A Nonasymptotic Theory of Gain-Dependent Error Dynamics in Behavior Cloning

1. Problem Statement

Behavior Cloning (BC) is a standard method for learning robot manipulation policies, typically outputting position targets that are executed by a low-level Proportional-Derivative (PD) controller. A critical empirical observation (noted by Bronars et al.) is that compliant-overdamped (low stiffness $K_p$, high damping $K_d$) controller gains yield higher closed-loop task success rates in BC, even though these settings often result in higher validation loss (worse open-loop prediction accuracy) compared to stiffer settings.

Existing theoretical frameworks for BC (e.g., Ross & Bagnell, DAgger) treat system dynamics as a black box, bounding error growth based on horizon length and per-step error without accounting for the specific structure of the controller. Consequently, there is no nonasymptotic theory explaining why "worse" prediction accuracy can lead to "better" closed-loop performance when specific controller gains are used.

2. Methodology

The paper bridges control theory and imitation learning by modeling the closed-loop error dynamics explicitly as a function of controller gains.

• System Modeling: The authors model the robot under a PD controller where the policy $\hat{\pi}_K$ outputs a position target $a_t$. The actual joint position $q_t$ is governed by the PD law. By linearizing around the expert trajectory, the position error dynamics are derived as a linear time-invariant (LTI) system driven by the policy's prediction error ($\xi_t$).
• Error Propagation: The prediction errors $\xi_t$ are modeled as independent, mean-zero, sub-Gaussian random vectors. The paper analyzes how these errors propagate through the closed-loop system matrices ($A_K, B_K$) to generate position errors ($e_t$).
• Discretization: The analysis accounts for the discrete-time nature of robot control using exact Zero-Order-Hold (ZOH) sampling, ensuring the theory applies to real-world sampled-data systems.
• Structural Assumptions: To derive scalar bounds, the authors introduce a "shape-preserving upper-bound structure," decomposing the error dynamics into three components: label difficulty, injection strength, and Lyapunov contraction.

3. Key Contributions

A. Sub-Gaussian Error Propagation (Theorem 1)

The paper proves that independent sub-Gaussian action errors propagate through the linear closed-loop dynamics to yield sub-Gaussian position errors.

• The "proxy matrix" $X_t(K)$, which governs the tail behavior of the position error, is derived as a gain-dependent Lyapunov sum.
• This establishes that the distribution of the position error is entirely determined by the controller gains and the statistics of the prediction error.

B. Closed-Loop Failure Bound (Theorem 3)

The authors derive a nonasymptotic upper bound on the probability of task failure over a horizon $T$.

• Factorization: The failure probability factorizes into two terms:
  1. Validation Loss + Generalization Slack: The standard prediction error metric.
  2. Gain-Dependent Amplification Index ($\Gamma_T(K)$): A term representing how much the controller amplifies prediction errors into position errors.
• Implication: A policy with higher validation loss can still have a lower failure probability if its associated amplification index is sufficiently small. This mathematically resolves the "paradox" of compliant gains outperforming stiff gains despite higher loss.

C. Scalar Ordering and Regime Comparison (Theorems 5 & 7)

Under shape-preserving assumptions, the proxy matrix is bounded by a scalar index $\Psi(K)$:
$$\Psi(K) = \frac{b(K) \cdot l(K)}{1 - \rho^*(K)^2}$$
Where:

• $l(K)$: Label difficulty (prediction error magnitude).
• $b(K)$: Injection strength (how aggressively errors enter the state).
• $\rho^*(K)$: Contraction rate (stability speed).

The paper defines four canonical gain regimes:

1. CO (Compliant-Overdamped): Low $K_p$, High $K_d$.
2. SU (Stiff-Underdamped): High $K_p$, Low $K_d$.
3. SO (Stiff-Overdamped): High $K_p$, High $K_d$.
4. CU (Compliant-Underdamped): Low $K_p$, Low $K_d$.

Result: The theory proves that CO yields the tightest (lowest) failure bound, while SU yields the loosest (highest) bound. The ordering between SO and CU is system-dependent.

D. Global Monotonicity for Canonical Systems (Theorem 9)

For a scalar second-order PD system, the authors derive a closed-form expression for the stationary variance of the position error:
$$X^c_\infty(\alpha, \beta) = \frac{\sigma^2 \alpha}{2\beta}$$
(where $\alpha$ is stiffness and $\beta$ is damping).

• This quantity is strictly increasing in stiffness ($\alpha$) and strictly decreasing in damping ($\beta$) across the entire stable region (both underdamped and overdamped).
• The exact ZOH discretization inherits this monotonicity, confirming that compliant-overdamped settings minimize error variance regardless of the specific damping regime.

4. Results and Validation

• Numerical Simulation: Using a scalar second-order system with 50,000 Monte Carlo rollouts, the authors verified that the Compliant-Overdamped (CO) regime resulted in position error envelopes roughly half the size of Stiff-Overdamped (SO) or Compliant-Underdamped (CU) regimes, and four times smaller than Stiff-Underdamped (SU).
• Failure Rates: Empirical failure rates at a specific tube radius confirmed the theoretical ordering: CO (1% failure) < CU (22%) $\approx$ SO (26%) < SU (75%).
• Bound Tightness: The theoretical failure bound (Theorem 3) was shown to be a conservative but qualitatively correct upper bound for the empirical failure rates across all regimes.

5. Significance

• Theoretical Resolution: This work provides the first rigorous, nonasymptotic explanation for the empirical success of compliant-overdamped controllers in Behavior Cloning. It demonstrates that minimizing prediction loss alone is insufficient; one must minimize the product of loss and the controller's error amplification.
• Design Guidelines: It offers a concrete criterion for gain selection: practitioners should tune gains to minimize the amplification index $\Gamma_T(K)$ (or the scalar proxy $\Psi(K)$), which naturally favors low stiffness and high damping.
• Bridging Fields: The paper successfully integrates sub-Gaussian concentration inequalities from learning theory with Lyapunov stability and H2 norms from control theory, creating a unified framework for analyzing learned control policies.
• Practical Impact: The findings suggest that in robot learning pipelines, "softer" controllers can be more robust to policy imperfections, guiding the design of safer and more reliable imitation learning systems.