Formation-Aware Adaptive Conformalized Perception for Safe Leader-Follower Multi-Robot Systems

Imagine a dance troupe where one dancer is the Leader and the rest are Followers. The Followers don't have a choreographer shouting instructions from the sidelines; instead, they must watch the Leader through their own eyes (cameras) and stay in a specific formation, like a V-shape or a line.

The biggest rule of this dance: The Leader must always stay in the Followers' field of view. If the Leader steps out of the camera frame, the dance breaks, and the robots might crash.

The Problem: "Blurry Glasses" and "Wobbly Steps"

In the real world, robots don't have perfect vision. Their cameras and AI brains make mistakes.

The Blur: Sometimes the robot thinks the Leader is 5 meters away, but they are actually 5.2 meters away.
The Wobble: The mistake isn't the same everywhere. If the Leader is right in the center of the camera, the robot sees them clearly (small error). But if the Leader is near the very edge of the camera's view (the "Field of View" or FOV), the vision gets shaky and the error gets huge.

The Old Way (The "One-Size-Fits-All" Approach):
Previous safety systems acted like a nervous parent who assumes the worst-case scenario every single time.

Analogy: Imagine a parent telling their child, "If you get within 1 inch of the edge of the pool, you might drown, so you must stay 10 feet away from the edge at all times."
Result: The robot stays so far away from the edge to be "safe" that it can't dance properly. It becomes stiff, slow, and often fails to keep the formation because it's too scared to move.

The Solution: "Smart, Adaptive Safety Glasses"

This paper introduces a new system called Formation-Aware Adaptive Conformalized Perception. Let's break that scary name down into a simple story.

1. The "Risk-Aware" Map (The Mondrian Map)

Instead of treating the whole camera view as equally dangerous, the robot divides the view into zones, like a painter's canvas (hence "Mondrian").

Green Zone (Center): The Leader is right in the middle. Vision is sharp. The robot knows, "I'm safe here. I can take a small margin of error."
Red Zone (Edge): The Leader is near the camera's edge. Vision is blurry. The robot knows, "This is dangerous! I need a huge safety buffer here."

2. The "Adaptive" Margin (The Stretchy Rubber Band)

The system uses a mathematical trick called Conformal Prediction to measure exactly how blurry the vision is in each zone.

In the Green Zone: It puts on a tight rubber band. It trusts its vision enough to get close to the edge, allowing for smooth, fast dancing.
In the Red Zone: It instantly stretches the rubber band wide open. It gives itself a massive safety cushion so that even if its vision is totally wrong, the Leader won't accidentally slip out of sight.

3. The "Smooth" Transition

The best part is that the robot doesn't snap from "tight" to "loose" like a light switch. It's like a dimmer switch. As the Leader moves from the center toward the edge, the safety buffer grows gradually. This prevents the robot from jerking around or getting stuck in a panic.

Why This Matters (The Results)

The researchers tested this in a computer simulation with real robot physics (Gazebo).

The Old Nervous Robot: Failed to keep the dance going 96% of the time because it was too scared to move near the edge.
The Old "Blind" Robot: Crashed or lost the Leader because it didn't account for the blurry vision at the edges.
The New "Smart" Robot: Succeeded 95% of the time. It danced beautifully in the center (because it wasn't overly cautious) and stayed safe near the edges (because it knew when to be extra careful).

The Takeaway

This paper teaches robots how to be smart about their own uncertainty. Instead of being paralyzed by fear of making a mistake, or reckless by ignoring it, they learn to adjust their safety rules based on where they are and how well they can see.

It's the difference between a driver who never drives on the highway because they are afraid of speed, and a driver who knows exactly how to slow down when the road gets foggy, but speeds up when the sun is shining.

Here is a detailed technical summary of the paper "Formation-Aware Adaptive Conformalized Perception for Safe Leader–Follower Multi-Robot Systems."

1. Problem Statement

The paper addresses the critical safety challenge in distributed vision-based leader–follower multi-robot systems. In these systems, follower robots rely on onboard perception (e.g., cameras) to estimate relative states (distance and bearing) to maintain formation and keep the leader within the camera's Field of View (FOV).

Key Challenges:

Heteroscedastic Perception Errors: Estimation errors are not uniform; they vary significantly based on the robot's state. Errors are typically small near the center of the image (low risk) but increase drastically near the FOV boundaries (high risk).
Coupling of Dynamics and Visibility: Formation maneuvers (turning, accelerating) directly impact visibility. If a follower loses the leader due to estimation errors, the system fails.
Limitations of Existing Methods:
- Nominal CBFs: Assume perfect state feedback, leading to safety violations when estimation errors occur near boundaries.
- Robust CBFs: Use worst-case error bounds, resulting in overly conservative behavior that restricts mobility even in safe regions.
- Standard Conformal Prediction (CP): Typically provides a single global error bound. This forces the system to adopt the worst-case margin everywhere, unnecessarily shrinking the feasible control set in low-risk configurations while potentially still being insufficiently conservative in high-risk, heavy-tailed error regimes.

2. Methodology

The authors propose a Formation-Aware Adaptive Conformalized Perception framework that integrates uncertainty quantification directly into the safety control layer.

A. Risk-Aware Mondrian Conformal Prediction (MCP)

Instead of a global bound, the method partitions the state space into risk regions based on the proximity to safety constraints (FOV limits).

State Partitioning: The state space is divided into $R$ $R$ risk groups ( $B_1, \dots, B_R$ $B_{1}, \dots, B_{R}$ ) using a scalar risk indicator $h(x)$ $h (x)$ (the minimum value of the safety barrier functions).
- High-risk: Near FOV boundaries.
- Low-risk: Near the image center.
Adaptive Quantiles: The calibration process computes separate nonconformity scores and quantiles ( $\hat{q}_{i,r}$ $\overset{q}{^}_{i, r}$ ) for each risk group.
- High-risk groups are assigned strict miscoverage levels ( $\delta_r$ small).
- Low-risk groups are assigned relaxed miscoverage levels ( $\delta_r$ large).
Smooth Transition: To avoid discontinuities in the control signal when switching between risk regions, a linear interpolation is applied over a transition buffer. This creates a continuous, state-dependent uncertainty margin $\tilde{q}_i(\hat{x}_i)$ .

B. Conformal Formation-Aware CBF-QP

The adaptive uncertainty bounds are integrated into a Control Barrier Function (CBF) Quadratic Program (QP).

Tightened Barrier: The safety constraint is tightened by subtracting the Lipschitz-scaled uncertainty margin from the nominal barrier function:
$\tilde{h}_\ell(\hat{x}_i) = h_\ell(\hat{x}_i) - L_{h_\ell} \cdot \tilde{q}_i(\hat{x}_i)$
Control Synthesis: At each time step, the controller solves a QP to minimize the deviation from the nominal tracking control while satisfying the tightened CBF constraints.
Theoretical Guarantee: Theorem 1 proves that this approach provides a probabilistic forward invariance guarantee. If the initial state satisfies the tightened constraints, the true state remains within the safe set with a probability of at least $1 - \sum \delta_r$.

3. Key Contributions

Adaptive Uncertainty Quantification: Introduction of a Risk-Aware Mondrian CP framework that conditions uncertainty bounds on the robot's formation state. This overcomes the "one-size-fits-all" conservatism of standard global CP.
Conformal CBF-QP Integration: Development of a Formation-Aware Conformal CBF-QP that utilizes smooth, state-dependent margins. This ensures strict FOV constraint satisfaction while maintaining optimization feasibility (avoiding the infeasibility issues of overly conservative global bounds).
Empirical Validation: Extensive Gazebo simulations demonstrating that the proposed method significantly outperforms both nominal controllers and static global CP baselines in terms of safety success rates and tracking accuracy.

4. Results

The system was tested in Gazebo simulations using a Husarion ROSbot 2 Pro (leader) and a Clearpath Jackal (follower) on an oval race track with three distinct risk regimes (low, medium, high).

Success Rate:
- Proposed Method: 95% success rate.
- Nominal Controller (No uncertainty): 4% (fails due to unmodeled errors).
- Low-Risk Global CP: 23% (unsafe near boundaries).
- High-Risk Global CP: 73% (too conservative, causing frequent QP infeasibility).
Tracking Performance: The proposed method achieved the best tracking accuracy in both distance and bearing angles. Unlike the high-risk global baseline, it did not sacrifice performance in low-risk regions.
Mechanism: The adaptive margin successfully shrank in low-risk zones (allowing aggressive tracking) and expanded near FOV limits (enforcing safety), as visualized in the simulation plots.

5. Significance

This work bridges the gap between rigorous safety guarantees and operational mobility in learning-enabled robotic systems.

Efficiency: It eliminates the unnecessary conservatism of global safety bounds, allowing robots to operate closer to their physical limits without compromising safety.
Scalability: The distributed nature of the framework allows each agent to compute safety locally based on its own perception and relative state.
Robustness: By explicitly accounting for the heteroscedastic nature of vision-based perception errors, the system provides a statistically valid safety certificate that adapts to the dynamic environment, making it highly suitable for real-world deployment in search and rescue, exploration, and collaborative transport tasks.