💻 computer science

DA-VPC: Disturbance-Aware Visual Predictive Control Scheme of Docking Maneuvers for Autonomous Trolley Collection

This paper proposes a Disturbance-Aware Visual Predictive Control (DA-VPC) scheme that integrates active infrared markers, nonholonomic kinematics modeling, and an extended state observer to achieve robust, high-precision docking maneuvers for autonomous trolley collection robots in diverse and disturbed environments.

Original authors: Yuhan Pang, Bingyi Xia, Zhe Zhang, Zhirui Sun, Peijia Xie, Bike Zhu, Wenjun Xu, Jiankun Wang

Published 2026-03-03

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Yuhan Pang, Bingyi Xia, Zhe Zhang, Zhirui Sun, Peijia Xie, Bike Zhu, Wenjun Xu, Jiankun Wang

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine a busy airport or a giant warehouse. Everywhere you look, there are shopping carts or luggage trolleys scattered about. Usually, humans have to walk around, grab them, and stack them up to clear the floor. This paper introduces a team of robots that can do this job all by themselves, but with a very specific and tricky challenge: they have to line up perfectly to push a whole row of carts at once.

Here is the story of how they solved this, explained simply:

The Problem: The "Blind" Push

Think of two robots working together. One robot (the Leader) is already at the front of the line, holding the first cart. The other robot (the Follower) has to drive up, grab the next cart, and push it gently until it clicks perfectly into place behind the first one.

This is harder than it sounds.

The "Non-Holonomic" Trap: Imagine you are driving a car. You can't just slide sideways; you have to turn the wheel to move forward. The robot is the same. If it tries to correct its angle too late, it might crash or miss the connection.
The "Slippery Floor" Problem: When the robot pushes a heavy cart, the wheels might slip on the floor, or the cart might wobble. It's like trying to walk on ice while carrying a heavy box; your brain thinks you are moving straight, but you are actually sliding sideways.
The "Glare" Issue: In places like airports, the floors are shiny, and the lights are bright. Standard cameras get confused by reflections, just like your eyes get blinded by a mirror.

The Solution: A "Super-Helper" System

The researchers built a system called DA-VPC (Disturbance-Aware Visual Predictive Control). Let's break down what that means using everyday analogies:

1. The "Glow-in-the-Dark" Beacons (Active IR Markers)

Instead of relying on the robot's camera to see the messy world around it, they put special infrared (IR) lights on the Leader robot.

The Analogy: Imagine the Leader robot is wearing a high-tech vest with glowing dots that only the Follower robot can see. To a human, the dots are invisible (like a remote control signal). To the robot, they are bright, clear targets that don't get confused by sunlight or shiny floors. This is like playing a game of "Red Light, Green Light" where only the robot can see the signal.

2. The "Crystal Ball" (Visual Predictive Control)

The robot doesn't just react to where it is now; it looks ahead.

The Analogy: Think of a basketball player shooting a free throw. They don't just look at the hoop; they calculate the arc, the wind, and where the ball will be in a second. This robot does the same. It simulates the next few seconds of movement in its brain to ensure that even if it has to turn, it won't lose sight of the glowing dots. It plans a smooth path to the finish line before it even starts moving.

3. The "Inner Ear" (Disturbance Observer)

This is the most clever part. The robot knows it might get pushed off course by a slippery wheel or a heavy cart.

The Analogy: Imagine you are walking with a friend who is holding a heavy box. You feel a sudden tug. Your "inner ear" (balance) instantly tells your brain, "Hey, something is pulling us!" and you adjust your step immediately to stay straight.
The robot has a digital version of this called an Extended State Observer (ESO). It constantly asks, "Am I moving exactly as I planned? If not, what is the invisible force pushing me?" It then instantly cancels out that force. If the wheels slip, the robot over-corrects immediately so the cart still lines up perfectly.

The Result: A Perfect Dance

The team tested this in real life:

In the dark: The robot worked perfectly because the glowing dots were the only thing that mattered.
In a parking lot with flickering lights: The robot ignored the chaos and focused on its targets.
On bumpy ground: Even when the robot slipped, the "Inner Ear" system fixed the mistake instantly.

They managed to stack carts with millimeter-level precision. It's like two people trying to stack a tower of Jenga blocks while one of them is on a moving boat, yet they never drop a block.

Why This Matters

This isn't just about robots stacking carts. It's about teaching machines to work in messy, unpredictable human worlds without needing expensive maps or GPS. By giving robots a way to "feel" when they are being pushed off course and a way to "see" clearly through the noise, we can finally have fleets of robots that can clean airports, warehouses, and malls efficiently, leaving humans to do the more creative work.

1. Problem Statement

The paper addresses the challenge of autonomous trolley collection and redistribution in public spaces (e.g., airports, warehouses). The specific task involves a "Leader-Follower" robot formation where a Follower robot must push a trolley to dock with a Leader robot (or a queue of trolleys) to form a rigid chain for transport.

Key Challenges Identified:

High Precision Requirements: Misalignment during docking can lead to collisions or failure to lock the trolleys together.
Environmental Disturbances: Real-world factors such as uneven terrain, wheel slippage, ground friction variations, and the variable load of the trolley introduce significant disturbances that degrade motion control.
Visual Limitations: Traditional visual markers (e.g., AprilTags) are susceptible to lighting interference and occlusion. Furthermore, LiDAR-based localization struggles in reflective environments (glass walls, polished floors).
Control Constraints: The system operates under nonholonomic constraints (differential drive) and a limited Field of View (FOV), making it difficult to simultaneously correct position and orientation errors without losing visual features.

2. Methodology

The authors propose a Disturbance-Aware Visual Predictive Control (DA-VPC) framework. The system operates without global localization or inter-robot communication, relying solely on monocular vision.

A. Hardware and Perception

Active IR Markers: The Leader robot is equipped with an array of active 940nm Infrared (IR) LEDs. The Follower robot uses a monocular camera with a narrowband filter to isolate these LEDs.
Robust Feature Extraction: The IR LEDs appear as stable, diamond-shaped speckles on the image plane, immune to ambient visible light changes. The system uses blob detection and image moments to extract sub-pixel centroids.
State Estimation:
- Feature Coordinates: Extracted via centroid calculation.
- Relative Orientation: Estimated using Homography Decomposition based on the coplanar arrangement of the IR markers.
- State Vector: Defined as $x = [s_0, ..., s_{n-1}, \theta_{F}^{L}]^T$ , combining image feature coordinates and the relative yaw angle between robots.

B. Control Framework

The control strategy integrates three main components:

Kinematic Modeling with Disturbance:
- The system models the Image-Based Visual Servoing (IBVS) dynamics, explicitly including a lumped disturbance term $d$ to account for unmodeled dynamics (e.g., trolley weight, slippage).
- Equation: $\dot{x} = J_x(x)u_F + d$ , where $J_x$ is the extended image Jacobian.
Extended State Observer (ESO):
- An ESO is designed to treat the total disturbance $d$ as an additional state.
- It estimates the disturbance in real-time ( $\hat{d}$ ) without requiring a precise dynamic model of the environment or the trolley.
- The observer ensures the estimation error is uniformly ultimately bounded.
Visual Predictive Control (VPC):
- A Model Predictive Control (MPC) scheme is employed to handle constraints (FOV limits, velocity bounds, and nonholonomic constraints).
- Optimization: The controller minimizes a cost function over a prediction horizon $N_p$ to drive the state error to zero while respecting constraints.
- Disturbance Compensation: The final control input is calculated by taking the nominal optimal control ( $\bar{u}^*$ ) and subtracting a compensation term derived from the ESO estimate:
  $u_F = \text{sat}(\bar{u}^* - J_x^\dagger \hat{d})$
- This feedforward compensation actively cancels out the estimated disturbances.

3. Key Contributions

Robust Monocular Docking System: A dual-robot system capable of precise docking using only monocular vision and active IR markers, eliminating the need for global localization or communication.
Disturbance-Aware Control: The integration of an Extended State Observer (ESO) with Visual Predictive Control to actively estimate and compensate for external disturbances (terrain, load) and internal modeling errors.
Constraint-Aware VPC: A novel control scheme that explicitly handles nonholonomic constraints and limited FOV, ensuring smooth convergence and preventing feature loss during the docking maneuver.
Active IR Feature Design: A hardware solution using 940nm IR LEDs that ensures reliable feature extraction in diverse lighting conditions (darkness, glare, flickering lights) where passive markers fail.

4. Experimental Results

The system was validated through simulations and extensive real-world experiments.

Simulation:
- Compared against basic IBVS and a previous nonholonomic VS method (Ke et al.).
- Results: The proposed DA-VPC achieved significantly lower position error ( $e_x \approx 1.43$ cm) and orientation error ( $e_\theta \approx 0.24^\circ$ ) compared to baselines. It successfully maintained smooth trajectories without the drift or feature loss observed in other methods.
Real-World Experiments:
- Disturbance Rejection: In experiments with rubber pads simulating uneven terrain, the DA-VPC achieved a lateral docking error of 0.4 cm, whereas the nominal VPC (without disturbance compensation) failed with an error of 4.6 cm.
- Sequential Docking: The robot successfully docked 8 trolleys in a queue over 80 trials. All lateral errors remained within the strict requirement of $\le 10$ cm (and $\le 6$ cm for the first trolley).
- Multi-Scenario Robustness: Successful docking was demonstrated in:
  - Dark outdoor environments (where passive markers fail).
  - Underground parking lots with flickering lights and irregular floors.
  - Reflective indoor environments (glass walls, polished floors) with sunlight interference.

5. Significance

This work represents a significant step toward the practical deployment of autonomous logistics robots in unstructured public spaces.

Reliability: By solving the critical issue of disturbance rejection in visual servoing, the system ensures high docking accuracy even in dynamic, unpredictable environments.
Cost-Effectiveness: The reliance on monocular vision and active IR markers offers a low-cost, scalable alternative to expensive LiDAR or multi-sensor fusion systems.
Scalability: The ability to stack multiple trolleys autonomously enables efficient "last-mile" or intra-facility logistics, reducing human labor in repetitive heavy-lifting tasks.

The paper concludes that while the current IR-based system excels at docking, future work will focus on integrating dynamic obstacle avoidance and whole-body control for more complex cooperative tasks.