Vision-based Goal-Reaching Control for Mobile Robots Using a Hierarchical Learning Framework

Technical Summary: Robust Vision-based Goal-Reaching Control for Mobile Robots Using a Hierarchical Learning Framework

Problem Statement
Reliable goal-reaching for large-scale mobile robots (e.g., 6000 kg skid-steered platforms) in harsh, partially known environments (mining, construction, forestry) remains challenging. While Reinforcement Learning (RL) offers high performance, its deployment on heavy robots is complicated by safety concerns, complex nonlinear actuator dynamics, and the risk of systematic faults. Existing RL approaches often assume perfect low-level actuation or are designed for lighter robots, failing to address the specific constraints of heavy in-wheel actuator chains, slip-induced disturbances, and the need for formal safety guarantees in real-time operation.

Methodology
The paper proposes a hierarchical, safety-aware control framework integrating four distinct layers:

1. Visual Localization: The system utilizes ORB-SLAM3 as a real-time stereo visual pose-estimation interface. It provides the robot's planar pose ($x, y, \theta$) to the planner and supervisor, featuring loop closing, map fusion, and relocalization capabilities to function without GNSS.
2. Constrained RL Motion Planner: A discrete Q-learning agent generates smooth, feasible motion references.
   • State & Action: The state space discretizes distance to goal, heading error, linear velocity, and angular velocity. Crucially, the agent outputs accelerations ($a_v, a_\omega$) rather than direct velocities to ensure smoothness.
   • Reward Shaping: The reward function combines task progress (distance reduction, timeout penalties) with specific shaping terms to suppress oscillations, enforce heading monotonicity, and penalize sign flips near the goal.
   • Policy Constraints: The planner incorporates hysteresis and zero-lock logic (clamping angular acceleration to zero when aligned with the goal) to eliminate residual heading wiggles typical of heavy skid-steered robots.
3. Actuator-Level Control (DNN + RAC):
   • Feedforward: A Scaled Conjugate-Gradient (SCG) trained Deep Neural Network (DNN) approximates a quasi-static feedforward map from desired wheel speed to nominal control input. This map is trained offline on data collected from asphalt and loose-soil terrains.
   • Feedback: A Robust Adaptive Controller (RAC) with a logarithmic barrier function compensates for residual modeling errors, slip, and bounded uncertainties. The control law ensures Uniformly Ultimately Bounded (UUB) tracking with exponential convergence to a disturbance-dependent residual set.
4. Supervisory Safety Logic: A sampled supervisor monitors the system for safety boundary violations (using a logarithmic barrier based on the distance to a safety circle) and localization inconsistencies (e.g., abnormal pose jumps). Upon detecting faults or unsafe conditions, it triggers a deterministic braking profile and switches the robot to a safe-return mode, guiding it back to a predefined inspection site.

Key Contributions
The authors emphasize that the novelty lies not in inventing new SLAM, RL, or control algorithms, but in their problem-specific integration and realization for a heavy off-road platform:

1. Integration: Coupling ORB-SLAM3, constrained Q-learning, SCG-trained DNNs, and RAC into a single control stack for a 6000 kg robot.
2. Constraint-Aware RL: Designing a Q-learning planner with acceleration-based actions, specific reward shaping for smoothness, and runtime logic (hysteresis/zero-lock) tailored to heavy skid-steered dynamics.
3. Actuator Realization: Implementing a learned DNN feedforward map combined with a logarithmic-barrier RAC to handle the nonlinearities and uncertainties of in-wheel actuator chains.
4. Safety Supervisor: Developing a state-machine supervisor that detects localization faults and enforces a safe-return trajectory, validated through fault-injection scenarios.
5. Experimental Validation: Validating the complete framework on a 6000 kg robot across asphalt and loose-soil terrains, demonstrating autonomous recovery after fault injection.

Experimental Results
Experiments were conducted on a 6000 kg skid-steered robot over 3.5 km of operation (50% asphalt, 50% soft soil).

• Goal Reaching: The system achieved a final-position Root Mean Square Error (RMSE) of approximately 3–4 cm (0.0317 m on asphalt, 0.0382 m on soft soil) in the SLAM estimation frame.
• Tracking Performance: The DNN-based RAC significantly outperformed baselines (Backstepping RAC and Model-based RAC) in terms of steady-state error, peak time, and maximum overshoot. The combination of DNN feedforward and adaptive feedback reduced side RMSE to 0.008 m/s, compared to ~0.039 m/s for feedforward alone.
• Safety & Fault Recovery: In fault-injection scenarios (simulated localization inconsistencies), the supervisor successfully detected the violation, initiated braking, and autonomously returned the robot to the safe inspection area.
• Ablation Studies: Removing hysteresis or zero-lock logic increased heading reversals and near-goal oscillations, confirming the necessity of these hand-engineered components for stabilizing the learned policy.

Significance and Claims
The paper claims to demonstrate the feasibility of a system-level integration for slow-speed goal-reaching on heavy robots. It highlights that the proposed framework successfully handles the complexities of heavy actuator chains and terrain variations while maintaining safety through a supervisory layer.

The authors are modest in their claims, explicitly stating:

• The framework is not a plug-and-play solution for arbitrary platforms; parameters and discretizations require retuning for different robots and terrains.
• The results validate performance under conservative speed and acceleration limits in obstacle-free environments; they do not claim high-speed autonomy or obstacle avoidance capabilities.
• The DNN feedforward map is valid within the recorded operating envelope; operation outside this envelope (e.g., different payloads or unseen terrains) is treated as bounded uncertainty handled by the RAC, not as a generalization of the DNN.
• The safety supervisor provides a deterministic response to specific detected faults but does not constitute a general proof of safety under all possible localization or terrain faults.

In conclusion, the work provides a validated pathway for deploying RL-based control on heavy industrial robots by bridging the gap between high-level planning and low-level actuator robustness, while ensuring safety through a dedicated supervisory layer.