🤖 AI

REAP: Reinforcement-Learning End-to-End Autonomous Parking with Gaussian Splatting Simulator for Real2Sim2Real Transfer

The paper proposes REAP, a Reinforcement Learning-based end-to-end autonomous parking system that leverages Soft Actor-Critic, behavior cloning, and a 3D Gaussian Splatting simulator to achieve efficient training and successful Real2Sim2Real transfer, particularly excelling in extreme scenarios like narrow mechanical parking slots.

Original authors: Changze Li, Zhe Chen, Shaoyu Chen, Lisen Mu, Yijian Li, Yuelong Yu, Qian Zhang, Qing Su, Ming Yang, Tong Qin

Published 2026-05-12

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Changze Li, Zhe Chen, Shaoyu Chen, Lisen Mu, Yijian Li, Yuelong Yu, Qian Zhang, Qing Su, Ming Yang, Tong Qin

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine teaching a car to park itself. Usually, we teach cars in two ways: either by showing them thousands of videos of humans parking (Imitation Learning), or by giving them a strict set of rules like "turn left, then go forward" (Rule-Based Planning).

But both methods have flaws. Showing videos is expensive and the car might just learn to "average" the movements, getting confused when the situation is slightly different. Strict rules often fail in tricky spots, like a tiny mechanical parking slot where there's only a few centimeters of space on each side.

This paper introduces REAP, a new way to teach cars to park using Reinforcement Learning. Think of this like training a dog: instead of showing it videos of other dogs, you let the car try, fail, get a "shock" (a penalty) if it hits something, and get a "treat" (a reward) when it parks perfectly. Over millions of tries, the car learns the best way to park on its own.

Here is how they made this work, broken down into simple concepts:

1. The "Magic Mirror" Simulator (Real2Sim2Real)

The biggest problem with training a car in a video game is that when you put it in the real world, it fails because the game looks too fake. The car sees a smooth, perfect wall in the game, but in reality, the wall is bumpy and dirty.

The authors built a special simulator using 3D Gaussian Splatting.

The Analogy: Imagine taking a photo of a real parking garage with a handheld scanner. Instead of building a fake 3D model out of blocks (like LEGO), they turned the actual photo into a cloud of millions of tiny, glowing dots that can be viewed from any angle.
Why it matters: This creates a "digital twin" of the real world that looks almost exactly like the real thing. They call this Real2Sim. They train the car in this hyper-realistic digital world, and because it looks so real, the car can drive into the actual parking lot (Sim2Real) without needing to relearn anything.

2. The "Smart Student" and the "Strict Teacher" (Asymmetric Learning)

Training a car to park is hard because it has to guess what's happening based on blurry camera images.

The Student (The Actor): This is the car's brain. It only sees what the cameras see (the "student" view). It has to figure out where to steer and how fast to go.
The Teacher (The Critic): This is the part that grades the student. In the simulator, the teacher has "X-ray vision." It doesn't just see the camera images; it sees the perfect, clean map of where the walls and cars are.
The Analogy: Imagine a student taking a test in a dark room (the car), while a teacher with a flashlight and a perfect map (the simulator) watches from above. The teacher knows exactly where the obstacles are and tells the student, "You're getting closer to a wall, slow down!" This helps the student learn much faster than if they were guessing blindly.

3. The "Safety Net" Rewards

In Reinforcement Learning, you have to tell the computer what a "good" move looks like.

The Problem: If you only say "Don't hit the wall," the car might learn to drive very close to the wall, just barely missing it, which is dangerous.
The Solution: They created a Soft Predictive Collision Penalty.
- The Analogy: Instead of just punishing the car when it crashes, they punish it for getting too close to a crash. It's like a video game where you lose points if you get near the edge of a cliff, not just when you fall off. This teaches the car to leave a nice, safe buffer zone around itself.

4. The "Cheat Sheet" (Behavior Cloning)

At the very beginning, the car knows nothing. If you let it explore randomly, it might crash a million times before learning anything.

The Solution: They let the car "cheat" at first. They have a simple, rule-based computer program that knows how to park. The car copies this program's moves (Behavior Cloning) to get started. As the car gets better, they slowly stop letting it cheat and force it to rely on its own Reinforcement Learning brain.

The Results: Can it actually work?

The team tested this on a real car in real parking lots.

Standard Spots: It worked great, parking successfully about 83% of the time in the simulation and 65–85% in the real world.
The "Impossible" Spot: The real test was a mechanical parking slot. These are tiny, narrow boxes with metal walls on both sides, leaving only about 10 centimeters (4 inches) of space on either side of the car.
- Old rule-based methods failed completely here.
- REAP managed to park in these tight spots about 55% of the time in the real world.

Summary

The paper claims that by using a hyper-realistic "digital twin" of the real world (3D Gaussian Splatting) and a smart training method that combines a "teacher" with a "student," they taught a car to park itself in extremely difficult, narrow spaces where traditional methods fail. They successfully moved the car from the computer simulation to a real vehicle without needing to retrain it.

Technical Summary: REAP – Reinforcement-Learning End-to-End Autonomous Parking with Gaussian Splatting Simulator

1. Problem Statement

Autonomous parking remains a significant challenge, particularly in extreme scenarios such as mechanical parking slots and dead-end spaces. Traditional multi-stage approaches (perception $\to$ planning $\to$ control) suffer from error accumulation and often fail when prior maps are unavailable or real-time perception is inaccurate. While end-to-end methods offer joint optimization, they face distinct hurdles:

Imitation Learning: Requires vast amounts of high-quality expert data, which is costly to produce and often leads to "averaging behavior" where the model fails to execute diverse or complex maneuvers effectively.
Reinforcement Learning (RL): While scalable, standard RL suffers from inefficient exploration and the "Sim2Real" gap. Models trained in simulators often fail when deployed in the real world due to discrepancies in visual fidelity and physical dynamics.
Simulation Limitations: Existing high-fidelity simulators (e.g., those using NeRF or traditional CAD assets) often lack the visual realism or physical accuracy required for wheeled robots, or they rely on manual modeling that is not scalable.

2. Methodology

The authors propose REAP (Reinforcement learning End-to-end Autonomous Parking), a vision-based system that predicts vehicle actions directly from surround-view camera images. The framework integrates three core components:

A. End-to-End Network with BEV Auxiliary Tasks

The system utilizes a unified Bird's-Eye View (BEV) representation. Surround-view images are transformed into BEV features, and a target parking slot is represented as a BEV heatmap.

Architecture: The network includes an actor (planner) and a critic.
Auxiliary Tasks: To enhance feature representation, the network is co-trained with three auxiliary perception tasks:
1. Occupancy (OCC): Predicts geometric occupancy to avoid obstacles.
2. Parking Slot Detection (PSD): Identifies available slots.
3. Target Prediction: Combines PSD with the user-selected target to ensure stable slot localization.
Modeling: Parking slots are modeled using 2D Gaussian distributions to maintain separability between adjacent spaces, and asymmetric gating functions distinguish accessible parking directions.

B. Asymmetric Hybrid Reinforcement Learning Strategy

To address the inefficiency of pure RL and the data dependency of imitation learning, REAP employs a dynamic hybrid strategy:

Behavior Cloning (BC): During exploration, the system queries a rule-based planner (Reed-Shepp curves) to generate expert actions. These are used to distill capabilities into the actor network via behavior cloning, providing immediate, state-matched guidance without pre-collected datasets.
Asymmetric Training: The actor learns from raw surround-view images (simulating real-world constraints), while the critic is trained with "privileged" information available only in simulation (artifact-free BEV maps). This stabilizes value estimation and reduces computational cost.
Soft Actor-Critic (SAC): The algorithm uses SAC to maximize both action value and policy entropy, encouraging exploration.
Replay Buffers: Two buffers are used: a primary buffer for all experiences and a "successful-experience" buffer specifically for collision-free successes, improving the utilization of rare positive samples in difficult scenarios.

C. Reward Design & Safety Mechanisms

The reward function ( $R_{total}$ ) combines sparse and dense rewards:

Sparse Rewards: Include success, timeout, boundary-crossing, and a novel Parking Anchor Reward. The anchor reward is granted when the vehicle reaches a state where a rule-based planner can generate a feasible trajectory, accelerating convergence in long-horizon tasks.
Dense Rewards & Safety:
- Predictive Collision Reward ( $R_{PC}$ ): Penalizes actions based on the ratio of collision length to total trajectory length, allowing the agent to distinguish between severe and mild potential collisions before they happen.
- Soft Collision Reward ( $R_{SC}$ ): Uses a Truncated Signed Distance Field (TSDF) to penalize actions that bring the vehicle too close to obstacles, encouraging a safe margin even if a hard collision does not occur.

D. Real2Sim2Real Simulator based on 3D Gaussian Splatting (3DGS)

To bridge the Sim2Real gap, the authors constructed a closed-loop simulator:

Real2Sim: Real-world scenes are captured using handheld LiDAR and cameras. 3D Gaussian Splatting (3DGS) is used to reconstruct the environment, providing photorealistic rendering and scale-accurate geometry without manual CAD modeling.
Sim2Real: The simulator incorporates domain randomization (lighting, camera intrinsics/extrinsics) and a 2.5D elevation map with a dual-layer vehicle model. This allows for accurate simulation of slopes, speed bumps, and the specific clearance constraints of mechanical parking slots (where the vehicle must pass over low barriers).

3. Key Contributions

REAP Algorithm: A vision-based end-to-end parking algorithm that significantly enhances performance in extreme scenarios. It combines behavior cloning for efficient initialization with asymmetric RL for robust policy learning.
3DGS-Based Simulator: A Real2Sim2Real pipeline that uses 3D Gaussian Splatting to create high-fidelity visual and physical simulations. This enables direct transfer of simulation-trained models to real vehicles, effectively narrowing the domain gap.
Soft Predictive Collision Penalty: A novel safety mechanism combining predictive collision assessment and soft distance-field penalties to reduce collision rates and encourage safe margins.
Real-World Deployment: Successful deployment on a physical vehicle platform, demonstrating the feasibility of end-to-end RL parking in extremely narrow mechanical slots (2.1m width with ~10cm clearance).

4. Experimental Results

The authors evaluated REAP across five scenarios: four standard parking garages (underground and ground-level) and one highly constrained mechanical parking slot.

Performance Metrics:
- Standard Slots: REAP-SAC achieved a 83.5% Parking Success Rate (PSR) in simulation and 65–85% in real-world tests.
- Mechanical Slots: In the most challenging scenario (2.1m width), REAP-SAC achieved a 70.0% PSR in simulation and 55.0% in real-world tests, with an average parking time of 89 seconds.
Comparisons:
- REAP outperformed rule-based planners (which often failed in complex maneuvers) and other RL/Imitation baselines (PPO, SAC, HOPE, ParkingE2E) in both success rate and collision avoidance.
- Ablation Studies: Removing the 3DGS simulator (replacing it with CAD or Mesh-based simulators) resulted in a drastic drop in real-world PSR (from 70% to 0% or 45%), confirming the critical role of visual fidelity. Removing auxiliary tasks or the hybrid training strategy also significantly degraded performance.
Sim2Real Gap: The close correlation between simulation and real-world results (e.g., 70% vs 55% in mechanical slots) validates the effectiveness of the Real2Sim2Real pipeline.

5. Significance and Claims

The paper claims that REAP represents a significant step forward in autonomous parking by:

Solving Extreme Scenarios: It is among the first end-to-end reinforcement learning approaches to successfully navigate and park in extremely narrow mechanical slots, a task where traditional methods often fail.
Efficient Real2Sim2Real Transfer: By leveraging 3DGS, the authors demonstrate that high-fidelity visual simulation can be achieved with minimal data collection costs (handheld scanning) and without manual modeling, enabling models trained in simulation to generalize directly to the real world.
Safety and Efficiency: The integration of soft predictive collision penalties and asymmetric RL training addresses the safety and exploration efficiency challenges inherent in vision-based RL.

Limitations Acknowledged by the Authors:

Smoothness: Fine-tuning improves success rates but can increase gear shifts, slightly reducing motion smoothness.
Training Dependence: The current pipeline still relies on rule-based planner supervision for early training stability and anchor reward definition.
Generalization: Validation is currently limited to structured environments; generalization to unseen outdoor environments or highly dynamic obstacles requires further study.
Computation: Deploying end-to-end models on edge devices introduces computational overhead, and the current safety design offers empirical rather than formal guarantees.