Fundamental Limits for Sensor-Based Control via the Gibbs Variational Principle

1. Problem Statement

The paper addresses the challenge of determining fundamental performance limits for feedback control systems operating under partial observations (noisy sensors).

• Context: Engineers need to know the minimum achievable cost (performance) for a given task to benchmark algorithms, select sensors, and certify task feasibility.
• Limitations of Existing Methods:
  • Classical tools (Nyquist-Shannon, Bode integrals) are often specific to linear or specific problem classes.
  • The existing TRIP framework (based on Fano's inequality) provides general bounds but suffers from two major flaws:
    1. It assumes bounded costs and finite control spaces.
    2. It calculates Mutual Information (MI) using the open-loop state distribution (the distribution without feedback). Since good feedback controllers concentrate the state in low-cost regions, the open-loop distribution vastly overestimates the information required, leading to loose (vacuous) bounds precisely when feedback is most effective.

2. Methodology

The authors derive a new lower bound by applying the Gibbs Variational Principle (GVP) from statistical mechanics to the joint path measure of states and observations.

A. The Gibbs Variational Principle (GVP)

The core mathematical tool is the inequality:
$$F_\beta[\ell] - \frac{1}{\beta} D[P \| Q] \leq E_P[\ell]$$
Where:

• $F_\beta[\ell]$ is the free energy (related to the partition function).
• $D[P \| Q]$ is the Kullback-Leibler (KL) divergence between the true path measure $P$ and a reference measure $Q$.
• $\beta$ acts as an inverse temperature.

The authors define a reference measure where sensor outputs are decoupled from the state. The KL divergence then quantifies the information utilized by the controller.

B. The TRFE Bound (Theorem 1)

They derive a Task-Relevant Free Energy (TRFE) lower bound, $J_{irr}(\bar{I})$, which represents the irreducible cost given a mutual information budget $\bar{I}$:
$$J_{irr}(\bar{I}) = \sup_{\beta > 0} \left\{ \inf_{u} \left( F_\beta^x(u) + J_u^{trj}(u) - \frac{\bar{I}}{\beta} \right) \right\} \leq J^\star$$

• Advantages over TRIP: It handles unbounded costs and continuous control spaces. It retains the full cost distribution rather than collapsing it to a binary threshold.

C. Self-Consistent Refinement (Theorem 2)

The paper's most significant innovation is recognizing a feedback loop between cost and information:

1. A controller achieving low cost must concentrate the state distribution.
2. A concentrated state distribution limits the maximum Mutual Information (MI) the sensor can extract (due to the entropy of the state).
3. A lower MI budget tightens the TRFE bound.

This leads to a fixed-point equation:
$$J = \Phi(J) = J_{irr}(\bar{I}(J))$$
Where $\bar{I}(J)$ is a "Cost-Information Certificate" derived from the cost structure (e.g., via second-moment constraints). The unique solution $J_{sc}$ to this equation is a provably tighter lower bound than the open-loop estimate.

D. Convexity and Computation (Theorem 3)

To ensure the bound is computable and certifiable:

• The authors identify conditions (matched actuator noise, quadratic control cost, semiconvexity of trajectory cost) under which the free energy minimization is strictly convex.
• Under these conditions, the numerical solution is guaranteed to be the global minimum, making the bound certifiably correct.
• Algorithm: They propose a numerical procedure using Monte Carlo rollouts to estimate the free energy across a grid of $\beta$ values, followed by a bisection search to solve the fixed-point equation.

3. Key Contributions

1. Free Energy Lower Bound (Theorem 1): A general lower bound on the minimum expected cost for causal feedback controllers under partial observations, applicable to nonlinear, nonholonomic, and hybrid dynamics with unbounded costs.
2. Self-Consistent Refinement (Theorem 2): A method to tighten the bound by coupling the achievable cost with the sensor's information capacity, resulting in a unique fixed-point solution that is strictly tighter than open-loop estimates.
3. Certifiable Computation (Theorem 3): Conditions under which the free energy minimization is convex, guaranteeing that the numerical bound is a valid lower bound on the optimal cost.
4. Practical Validation: Demonstration on a nonlinear Dubins car tracking problem where the self-consistent bound remains informative even at low sensor noise, whereas open-loop bounds become vacuous.

4. Results

The authors validated their method on a nonlinear Dubins car tracking problem (tracking a figure-eight path with unobserved heading).

• Setup: The system has process noise in the steering channel and noisy position measurements.
• Comparison:
  • LQG (Upper Bound): A Kalman filter + LQR controller (linearized) serves as a practical upper bound on performance.
  • OL-MI (Open-Loop Bound): The TRFE bound using the open-loop MI budget.
  • SC (Self-Consistent Bound): The proposed fixed-point solution.
• Findings:
  • At low sensor noise ($\sigma_v = 0.1$), the OL-MI bound was vacuous (negative), while the SC bound captured ~10% of the optimal cost.
  • At moderate noise ($\sigma_v \approx 2$), the SC bound captured >50% of the LQG cost.
  • As noise increased ($\sigma_v \to \infty$), all bounds converged to the open-loop cost, confirming tightness in the no-information limit.
  • The SC bound was uniformly tighter than the OL-MI bound across all noise levels.

5. Significance

• General-Purpose Tool: Unlike classical control limits (e.g., Bode integrals) that are restricted to linear systems, this approach works for nonlinear, nonholonomic, and hybrid systems.
• Sensor Selection & Feasibility: It provides a rigorous way to determine if a task is feasible with a specific sensor modality before designing a controller.
• Bridging Fields: It establishes a deep connection between stochastic optimal control and statistical mechanics (thermodynamics of information), suggesting that the "cost of control" is fundamentally linked to the "physics of information processing."
• Overcoming Structural Flaws: By moving away from the open-loop state distribution assumption, the method avoids the structural weakness of previous information-theoretic bounds, providing meaningful limits even when feedback is highly effective.