Stochastic Optimal Feedforward-Feedback Control for Partially Observable Sensorimotor Systems

This paper introduces a computationally tractable framework for stochastic optimal control in partially observable, nonlinear systems that explicitly integrates feedforward planning with feedback uncertainties, demonstrating how muscle co-contraction in human neuromechanics emerges as an optimal adaptation to sensorimotor constraints.

The Gist

Imagine you are trying to walk across a tightrope in a hurricane. You have two main ways to stay balanced:

1. The "Stiff" Strategy (Feedforward): You tighten your core muscles and grip the rope so hard that your body becomes a rigid, unyielding pole. You don't wait to feel the wind; you just brace yourself against the chaos.
2. The "Wobbly" Strategy (Feedback): You stay loose and wait for the wind to push you. As soon as you feel yourself tipping, you quickly shift your weight to correct it.

The problem? In the real world (and in our brains), there is a delay. It takes time for your eyes to see the wind, for your brain to process it, and for your muscles to react. If the wind is too strong or the delay is too long, the "Wobbly" strategy fails because you react too late and fall.

This paper introduces a new mathematical "rulebook" that explains how the brain (and robots) figure out the perfect mix of Stiffness and Wobble to stay safe, even when the world is noisy and unpredictable.

The Big Problem: The Brain's "Lag"

The authors point out that our brains are slow. By the time you feel your arm move, it's already too late to fix a small error if the situation is chaotic. Traditional computer models often try to solve this by either:

• Ignoring the noise (pretending the world is perfect).
• Using simple, linear math that breaks down when things get complex (like a human arm with six muscles).

The authors say, "We need a way to plan ahead while knowing that our sensors are flawed and slow."

The Solution: A "Crystal Ball" with a Safety Net

The authors created a new framework that combines two things:

1. The Plan (Feedforward): A pre-planned path, like a GPS route.
2. The Safety Net (Feedback): A system to correct mistakes, but one that knows exactly how "foggy" the sensors are.

They used a clever trick called Statistical Linearization. Think of this like a weather forecast. You can't predict the exact path of every single raindrop (too complex), but you can predict the average rain and how much it might vary (the statistics).

By using this "average + variation" approach, they turned a terrifyingly complex, chaotic math problem into a manageable one. They didn't lose the "realness" of the situation (the non-linearities); they just found a way to calculate the best path through the chaos without needing a supercomputer the size of a building.

What They Discovered: The "Muscle Hugging" Secret

They tested this theory on human arm movements, specifically looking at muscle co-contraction. This is when you tighten opposing muscles at the same time (like flexing your bicep and tricep together).

The Old View: Scientists used to think co-contraction was just a clumsy, energy-wasting habit we do when we are scared or unsure.

The New View (from this paper): Co-contraction is actually a brilliant, optimal strategy.

• When the world is quiet (low noise): Your brain says, "Relax! I can see clearly. I'll use my feedback loop to make tiny, precise corrections." You use very little muscle stiffness.
• When the world is chaotic (high noise or bad vision): Your brain says, "The sensors are too slow and foggy! I can't rely on corrections." So, it switches to the Stiff Strategy. It tightens the muscles to create a rigid, stable "exoskeleton" that resists the chaos automatically.

The Analogy:
Imagine driving a car.

• On a sunny day with clear roads (low noise), you drive loosely, making small steering adjustments as you go.
• In a blizzard with zero visibility (high noise), you don't wait to see the edge of the road to steer. You lock your steering wheel, press the gas, and trust the car's suspension to keep you on the path. You are "co-contracting" your driving style.

Why This Matters

This isn't just about human arms. This framework is a universal tool for:

• Robots: Helping robots with "soft" arms (variable impedance actuators) navigate messy, real-world environments without falling over.
• Medicine: Understanding why people with certain neurological conditions might stiffen up (co-contract) when they are anxious or in pain. It's not a malfunction; it's their brain's optimal solution to a noisy system.
• AI: Moving beyond "black box" AI that just learns by trial and error, toward systems that understand why they make certain decisions based on uncertainty.

The Takeaway

The brain is not a perfect calculator; it's a master of managing uncertainty. When the future is foggy and our senses are slow, the smartest thing to do isn't to try harder to see better—it's to brace yourself, tighten up, and let your body's natural physics do the heavy lifting. This paper gives us the math to prove that "bracing" is actually the most intelligent move you can make.

Technical Summary

Here is a detailed technical summary of the paper "Stochastic Optimal Feedforward-Feedback Control for Partially Observable Sensorimotor Systems" by Berret and Jean.

1. Problem Statement

The paper addresses the fundamental challenge of controlling complex, high-dimensional, nonlinear, and stochastic systems that are partially observable and subject to sensory delays.

• Context: In both engineered systems and biological motor control (e.g., human movement), pure feedback control is insufficient due to systemic latencies (e.g., >50 ms in humans) and sensory noise.
• The Gap: While the Central Nervous System (CNS) successfully combines feedforward mechanisms (anticipatory muscle co-contraction) with feedback corrections, existing mathematical frameworks struggle to derive optimal control policies for such systems.
  • Standard Linear-Quadratic-Gaussian (LQG) models are too restrictive (linear dynamics).
  • Classical stochastic optimal control involves infinite-dimensional equations in belief space, making them computationally intractable.
  • Existing numerical methods (e.g., iLQG, Stochastic DDP) often struggle to seamlessly integrate state estimation with planning under uncertainty or lack theoretical guarantees on approximation fidelity.

Goal: Develop a computationally tractable, continuous-time framework to determine optimal control policies that explicitly account for feedback uncertainties, latencies, and nonlinear dynamics, while preserving interpretability.

2. Methodology

The authors propose a novel framework that extends Neighboring Optimal Control by integrating the planning phase with stochasticity, rather than treating them sequentially.

A. Problem Formulation

The system is modeled as a partially observable stochastic control system:

• State Dynamics: Governed by a stochastic differential equation (SDE) with nonlinear drift $f$ and diffusion $G$.
• Observations: Partial and noisy, subject to delays and integration processes (modeled via auxiliary variables).
• Control Policy: Restricted to an affine dependence on the state estimate:
  $$u_t = u_{ol}(t) + L(t)(\hat{x}_t - m_{ol}(t))$$
  Where $u_{ol}$ is the open-loop (feedforward) plan, $L$ is the feedback gain, $\hat{x}_t$ is the state estimate (from a Kalman-type filter), and $m_{ol}$ is the nominal deterministic trajectory.

B. Statistical Linearization

The core innovation is the use of Statistical Linearization to transform the stochastic problem into a deterministic one:

1. Augmented State Space: The problem is reformulated in a space containing the mean trajectory ($m$), the error covariance ($P$), and an auxiliary matrix ($S$) related to the Riccati equation.
2. Approximation: By assuming the cost function is quadratic (or can be approximated as such), the authors show that the expected cost $J(u)$ depends only on the first two moments (mean and covariance) of the state distribution.
3. Deterministic Equivalent: The original stochastic optimization is approximated by a deterministic optimal control problem in an augmented state space of dimension $n(n+2)$.
   • This new problem retains the nonlinearities of the original drift $f$ (unlike standard linearization).
   • It allows the use of state-of-the-art deterministic trajectory optimization tools (e.g., Direct Transcription with implicit Euler).

C. Handling Delays

Sensory delays are explicitly modeled by introducing auxiliary state variables ($z_t$) that represent the "integrated" sensory signal. This transforms the delay differential equations into a standard SDE within the augmented state vector, allowing the Kalman filter to estimate the true state despite the lag.

3. Key Contributions

1. Unified Framework: A continuous-time framework that unifies feedforward planning and feedback control, explicitly accounting for stochasticity during the planning phase.
2. Theoretical Guarantees: Unlike many data-driven or heuristic approaches, this method provides theoretical error estimates based on statistical linearization, ensuring the approximation fidelity is quantifiable.
3. Nonlinearity Preservation: The method preserves the critical nonlinear properties of the system dynamics (e.g., activation-dependent viscoelasticity), allowing the controller to exploit these nonlinearities for robustness.
4. Computational Tractability: It reduces a high-dimensional, partially observable stochastic problem to a finite-dimensional deterministic optimization problem solvable with standard numerical tools.

4. Results & Applications

The framework was applied to two human neuromechanical tasks to validate its predictions against empirical observations.

A. Forearm Stabilization Task (1-DOF)

• Setup: Stabilizing an unstable forearm using antagonist muscles (flexors/extensors) under varying levels of sensory noise ($\eta$) and delay ($\rho$).
• Findings:
  • High Noise/Delay: The optimal strategy shifts heavily toward feedforward muscle co-contraction (increasing joint stiffness) and reduces reliance on feedback gains. Co-contraction provides intrinsic stability, mitigating the need for error-prone feedback corrections.
  • Low Noise/Delay: The system relies more on state feedback control (reciprocal muscle activation) to correct errors, with minimal co-contraction.
  • Cost: While co-contraction is metabolically costly, it becomes the optimal strategy when sensory uncertainty is high, minimizing the total expected cost (error + effort).

B. Planar Reaching Task (2-DOF, 6 Muscles)

• Setup: Reaching movements in a Divergent Force Field (DF) (unstable environment) vs. a Null Field (NF), with and without visual feedback.
• Findings:
  • Interaction of Dynamics and Sensory Noise: In the DF condition, the absence of vision (high noise) led to the highest levels of muscle co-contraction, particularly in biarticular muscles.
  • Robustness: With high noise, the optimal policy relies almost entirely on feedforward stiffness to stabilize the trajectory, rendering online feedback corrections negligible.
  • Adaptation: The model successfully predicted that co-contraction is not just a "brute force" response but an optimal adaptation to the specific combination of environmental instability and sensory uncertainty.

5. Significance and Impact

• Neuromechanics: The paper provides a computational foundation for understanding muscle co-contraction. It resolves the long-standing debate between impedance control (feedforward) and state feedback control by showing they are not mutually exclusive but are optimally balanced based on the quality of sensory information.
• Robotics: The framework offers a generalizable tool for designing robust controllers for variable impedance actuators in nonlinear robotic systems, particularly those operating in uncertain environments with sensor delays.
• Theoretical Advancement: It bridges the gap between rigorous stochastic control theory and practical trajectory optimization, offering a method that is both theoretically grounded (with error bounds) and computationally feasible for high-dimensional systems.

In summary, Berret and Jean demonstrate that muscle co-contraction is an optimal, mathematically derived adaptation to sensory uncertainty and latency, emerging naturally when planning under uncertainty is integrated with feedback control in a nonlinear stochastic framework.