🤖 AI

RAY-TOLD: Ray-Based Latent Dynamics for Dense Dynamic Obstacle Avoidance with TDMPC

This paper proposes RAY-TOLD, a hybrid control framework that integrates LiDAR-based latent dynamics and a policy mixture strategy into MPPI to enhance autonomous robot navigation in dense, dynamic crowds by combining short-horizon physical robustness with long-horizon learned foresight to reduce collision rates and escape local minima.

Original authors: Seungho Han, Seokju Lee, Jeonguk Kang

Published 2026-05-01

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Seungho Han, Seokju Lee, Jeonguk Kang

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to walk through a crowded, chaotic dance floor where people are moving unpredictably, bouncing off walls, and changing directions suddenly. Your goal is to get to the other side without bumping into anyone.

This is exactly the challenge autonomous robots face. The paper introduces a new "brain" for robots called RAY-TOLD that helps them navigate these dense crowds much better than current methods.

Here is how it works, broken down into simple concepts:

The Problem: The "Short-Sighted" Robot

Current robots often use a method called MPPI. Think of MPPI as a robot that looks only 3 seconds into the future.

How it works: It simulates thousands of possible paths in its head for the next few seconds and picks the safest one.
The Flaw: Because it only looks a short distance ahead, it often gets stuck. Imagine walking toward a goal, but a crowd blocks your path. The robot sees a safe path right now that leads into a dead end (like a U-shaped wall or a tight cluster of people). It doesn't realize the dead end until it's too late because it can't "see" far enough ahead to know that the path leads nowhere. It gets trapped in a "local minimum"—a safe spot that isn't the goal.

The Solution: RAY-TOLD (The "Experienced Guide")

The authors created RAY-TOLD, which is a hybrid system. It combines the robot's immediate reflexes with a "learned intuition" about the future.

Think of it as giving the robot two superpowers:

The "Latent Map" (The Compressed Memory):
Instead of trying to process every single laser beam from its sensors (which is like trying to read every word in a library to find one book), RAY-TOLD compresses all that sensor data into a compact, low-dimensional "mental map."
- Analogy: Imagine looking at a dense forest. Instead of counting every leaf, your brain instantly recognizes "thicket," "clearing," or "path." RAY-TOLD does this with laser scans, turning complex data into a simple, easy-to-understand picture of where the danger is.
The "Policy Prior" (The Experienced Guide):
This is the secret sauce. The robot has been trained to learn a "policy"—essentially, a set of habits or instincts about how to move toward a goal in a crowd.
- The Innovation: When the robot plans its next move, it doesn't just guess randomly. It asks its "Experienced Guide" (the learned policy) for suggestions.
- The Mix: The system uses a mixture strategy. It takes 80-90% of its path ideas from random, safe, physics-based guesses (to ensure it doesn't crash immediately), but it injects 10-20% of ideas directly from the "Experienced Guide."
- Why this helps: The random guesses keep the robot safe right now, but the "Experienced Guide" nudges the robot toward paths that lead to the goal later, preventing it from getting stuck in dead ends.
The "Crystal Ball" (Terminal Value):
Standard robots stop thinking once their 3-second horizon is up. RAY-TOLD adds a "crystal ball" at the end of that 3-second window. It asks, "If I end up at this spot in 3 seconds, how good is my situation for the rest of the journey?" This prevents the robot from choosing a path that looks safe now but is a dead end later.

How They Tested It

The researchers put their robot in a simulated room with 40 to 60 moving obstacles (like a very crowded party). These obstacles moved randomly, bounced off walls, and changed direction.

The Old Way (MPPI): The robot got stuck or crashed about 11% of the time. It often couldn't find a way through tight, converging crowds.
The New Way (RAY-TOLD): By using the "Experienced Guide" to nudge its decisions, the robot succeeded 94% of the time and crashed only 6% of the time.

The "Sweet Spot"

The paper found a perfect balance.

If the robot relied too much on the "Experienced Guide" (too much guidance), it sometimes made bad guesses because the guide wasn't perfect for every weird situation.
If it relied too little, it acted like the old, short-sighted robot.
The Winner: A mix where the robot follows its own physics-based safety checks most of the time, but lets the "Experienced Guide" steer it just enough to avoid dead ends.

Summary

RAY-TOLD is like giving a robot a pair of glasses that let it see the "big picture" of a crowded room while still keeping its feet firmly on the ground. It combines the safety of immediate reflexes with the wisdom of long-term planning, allowing robots to navigate dense, chaotic crowds without getting stuck or crashing.

1. Problem Statement

Autonomous mobile robots face significant challenges when navigating dense, dynamic crowds. Existing approaches suffer from specific limitations:

Purely Reactive Methods (e.g., MPPI): Model Predictive Path Integral (MPPI) control is robust for short-term kinematic feasibility and collision avoidance. However, it relies on a limited prediction horizon. In complex scenarios (e.g., U-shaped obstacles or converging crowds), MPPI often gets trapped in local minima because it lacks long-term spatial awareness and cannot anticipate future obstacle behaviors beyond its short rollout window.
Pure Reinforcement Learning (RL): While RL excels at long-horizon reasoning, end-to-end model-free approaches often struggle to enforce hard safety constraints and suffer from inefficient exploration in safety-critical physical systems.
The Gap: There is a need for a hybrid architecture that combines the safety and kinematic constraints of physics-based MPC with the long-horizon foresight of learned models to navigate stochastic, high-density environments.

2. Methodology: RAY-TOLD

The authors propose RAY-TOLD (Ray-based Task-Oriented Latent Dynamics), a hybrid control framework that integrates LiDAR-centric latent dynamics with sampling-based MPPI.

A. Core Architecture

The system operates in two parallel modalities:

Physics-Based Planner (MPPI): Handles short-horizon ( $H$ ) trajectory rollouts using a differentiable kinematic bicycle model to ensure immediate kinematic feasibility and collision avoidance.
Latent Dynamics Model (TOLD): A learned world model that encodes high-dimensional sensor data into a compact latent space to provide long-horizon guidance.

B. Key Components

State Representation: The input state vector $x_t$ includes vehicle kinematics ( $s_t$ ), relative goal position ( $s_{goal}$ ), and LiDAR measurements ( $s_{ray}$ ). The LiDAR data is "occlusion-aware," meaning rays are truncated at the first obstacle hit, simulating real-world line-of-sight constraints.
Latent Encoder & Dynamics:
- An encoder $h_\theta$ compresses high-dimensional inputs into a low-dimensional latent state $z_t$ .
- A latent dynamics model $d_\phi$ predicts state transitions in the latent space.
- A reward predictor $R_\psi$ and a Terminal Value Function $C_\omega$ are trained to estimate immediate rewards and infinite-horizon returns, respectively.
Policy Prior ( $\pi_\xi$ ): A policy network trained to propose goal-directed actions within the latent space.

C. Policy Mixture Sampling Strategy

The core innovation lies in how RAY-TOLD modifies the MPPI sampling process to overcome local minima:

Standard MPPI: Samples actions purely from a Gaussian distribution centered on the previous solution (unbiased noise).
RAY-TOLD: Uses a mixture sampling strategy. A fraction $\alpha$ $α$ of the $K$ $K$ candidate trajectories are seeded with actions derived from the learned policy prior $\pi_\xi$ $π_{ξ}$ (unrolled in latent space), while the remaining $(1-\alpha)$ $(1 - α)$ are sampled via standard MPPI noise.
- Formula: $a^{(k)} \sim \mathcal{N}(a^\pi, \Sigma)$ if $k < \alpha K$ , else $\mathcal{N}(a^{MPPI}, \Sigma)$ .
Evaluation: The cost of each trajectory is calculated as the sum of short-horizon rewards plus a learned terminal value $C_\omega(z_{t+H})$ , which estimates the cost-to-go beyond the planning horizon.

3. Key Contributions

Hybrid Architecture: Seamless integration of LiDAR-centric latent dynamics with MPPI, enabling the robot to navigate dense dynamic crowds by leveraging both physics-based constraints and learned intent.
Policy Mixture Sampling: A novel technique that augments the MPPI candidate population with learned policy trajectories. This eliminates MPPI's vulnerability to local minima without requiring handcrafted heuristics.
Terminal Value Integration: The use of a learned terminal value function allows the planner to "look ahead" beyond the physical rollout horizon, effectively guiding the robot out of dead ends.
Empirical Validation: Extensive testing in highly stochastic environments demonstrating significant reductions in collision rates compared to standard baselines.

4. Experimental Results

The method was evaluated in a 20m $\times$ 10m environment with 40–60 dynamic obstacles moving according to a modified Social Force Model (SFM).

Baseline Performance: Standard MPPI achieved an 89.0% success rate with an 11.0% collision rate. It frequently failed in scenarios involving converging obstacles or reflections off walls due to sample sparsity.
RAY-TOLD Performance:
- RAY-TOLD ( $\alpha=0$ ): (Value function only, no policy guidance) Achieved the same success rate as MPPI but improved the safety margin, validating the value function's ability to assess terminal states.
- RAY-TOLD ( $\alpha=0.1$ ): (Optimal Mix) Achieved the highest success rate of 94.0% and reduced the collision rate to 6.0% (a ~45% improvement over baseline). This configuration balanced long-horizon guidance with short-horizon reactivity.
- RAY-TOLD ( $\alpha=0.2$ ): (Strong Guidance) Success rate dropped slightly to 91.0%, suggesting that over-reliance on the policy prior can introduce bias in highly stochastic edge cases.
Qualitative Analysis:
- Fig. 3: Visualizations show RAY-TOLD successfully navigating converging obstacles and anticipating wall reflections (where MPPI failed).
- Fig. 4 (t-SNE): The latent space visualization reveals a smooth topological transition from "dangerous" (red) to "safe" (blue) states, proving the model learns physically meaningful representations of collision risk rather than memorizing raw sensor data.

5. Significance

RAY-TOLD represents a significant advancement in robotic navigation by solving the "local minima" problem inherent in reactive planners without sacrificing safety.

Safety & Reliability: By grounding the search in physics-based rollouts while injecting learned long-term intent, the system ensures kinematic feasibility while avoiding myopic decisions.
Generalization: The use of a latent world model allows the robot to generalize to unseen obstacle configurations and dynamic behaviors (e.g., reflections) rather than memorizing specific paths.
Practicality: The method operates at 50 Hz in a parallelized simulation, demonstrating feasibility for real-time deployment in human-centric environments.

In conclusion, RAY-TOLD effectively bridges the gap between the robustness of Model Predictive Control and the foresight of Reinforcement Learning, offering a robust solution for autonomous navigation in the most challenging, dense, and dynamic environments.