Optimal feedback control under stepwise equilibration and partial observation

Technical Summary: Optimal Feedback Control under Stepwise Equilibration and Partial Observation

Problem Statement
The paper addresses the problem of minimizing the work required to drive a mesoscopic system through a finite-time transformation using feedback control. While open-loop protocols for such transformations are well-characterized by thermodynamic length in slow-driving regimes, the corresponding feedback problem—constructing optimal finite-horizon closed-loop protocols under partial observation—remains less understood. Existing solutions often rely on approximations (truncated belief states, learned policies) or are restricted to specific cases (single measurements, passive particles). The authors focus on systems operating in a "stepwise equilibration" regime, where control actions are triggered by noisy signals that are statistically correlated with the microscopic state, and the system fully equilibrates between rapid, measurement-conditioned changes to its energy landscape.

Methodology
The authors model the control problem as a Partially Observable Markov Decision Process (POMDP) within a specific structural limit: the thermalization–observation–quench cycle.

1. Stepwise Equilibration Limit: The protocol consists of $N$ discrete cycles. In each cycle, the system relaxes to the equilibrium distribution of the current Hamiltonian $H_n$, an observation $o_n$ is acquired, and an instantaneous, observation-conditioned quench updates the Hamiltonian to $H_{n+1}$.
2. State Reduction: Because the system equilibrates at every decision step, the memory of previous interventions is erased. The only state variable propagated by the controller is the Hamiltonian itself (or its parameters), rather than a complex belief distribution over non-equilibrium states.
3. Bellman Recursion: This structural simplification reduces the optimal control problem to an exact finite-horizon Bellman recursion over observation-conditioned Hamiltonian updates. The value function $V_r(H)$ represents the minimum expected remaining work to reach a target Hamiltonian $H_f$ in $r$ steps.
4. Analytical Solution: The authors apply this framework to a canonical benchmark: a one-dimensional harmonic trap with fixed stiffness $k$, translated from $\lambda=0$ to $\lambda=L$. The observation channel is translation-covariant (e.g., additive Gaussian noise). They exploit the quadratic structure of the cost and the Gaussian nature of the equilibrium distribution to solve the recursion analytically.

Key Contributions and Results

1. Exact Optimal Feedback Law:
   For the translated harmonic trap, the authors derive a closed-form optimal feedback law. The optimal trap center $\lambda^*$ at any step is a horizon-weighted compromise between the controller's estimate of the microstate $\hat{x}$ and the target $L$:
   $$\lambda^*(o; r) = \frac{r-1}{r}\hat{x}(o; \lambda) + \frac{1}{r}L$$
   where $r$ is the number of remaining cycles. Early in the protocol (large $r$), the controller tracks the estimated fluctuation to extract work; as the deadline approaches (small $r$), the controller shifts focus to enforcing the endpoint constraint.

2. Transport–Information Decomposition of Work:
   The minimum expected work $W^*_N$ is shown to decompose into two distinct terms:
   $$W^*_N = \underbrace{\frac{k_B T}{2} \frac{\mathcal{L}^2}{N}}_{\text{Thermodynamic-length transport cost}} - \underbrace{\frac{k_B T}{2} \chi (N - H_N)}_{\text{Observation-enabled extraction}}$$

   • The first term is the geometric cost of transporting the equilibrium distribution, recovering the open-loop optimum based on thermodynamic length $\mathcal{L} = \sqrt{\beta k} L$.
   • The second term is a negative "extraction" term proportional to $\chi$, the fraction of equilibrium fluctuations resolved by the measurement (signal-to-noise ratio).
   • This decomposition reveals a crossover: for a small number of cycles, transport costs dominate (dissipative); for a large number of cycles, the information-enabled extraction exceeds transport costs, resulting in "bare" negative work (apparent harvesting).

3. Optimal Cycle Count with Overhead:
   When a fixed physical cost $c$ (overhead) is incurred per intervention (measurement, actuation, fuel), the total cost $J_N = W^*_N + cN$ is minimized at a finite number of cycles $N^*$. The authors derive an explicit design rule for $N^*$ based on the dimensionless overhead $\hat{c} = 2c/k_B T$ and the resolution $\chi$. A finite optimum exists only if the overhead exceeds the asymptotic per-cycle extraction rate ($\hat{c} > \chi$).

4. Thermodynamic Feasibility and Landauer's Bound:
   The paper critically examines the "apparent harvest" regime where $W^*_N < 0$. By accounting for the thermodynamic cost of creating and resetting the measurement record (Landauer's bound), the authors show that the minimum cost of information processing per cycle is $k_B T I(x; o)$.

   • For a Gaussian channel, the mutual information $I(x; o)$ sets a lower bound on the overhead that is strictly greater than the extraction threshold $\chi$.
   • Consequently, once the cost of resetting the memory is included, the net work is always non-negative. The "unlimited harvest" regime is thermodynamically inaccessible. The apparent negative work in the bare calculation is revealed as a bookkeeping artifact of pricing observations at zero.

Significance
The paper claims to provide the first analytically solvable instance of a general structure for optimal feedback control in the stepwise-equilibration limit. Its significance lies in:

• Exactness: Reducing a generally intractable POMDP to a solvable finite-horizon Bellman recursion where the state is simply the Hamiltonian.
• Physical Insight: Explicitly separating the geometric transport cost from the information-enabled extraction, clarifying the conditions under which feedback appears to harvest work.
• Thermodynamic Consistency: Demonstrating that when the full thermodynamic cost of information processing (Landauer cost) is included, the net work of such protocols is non-negative, and a finite optimal number of feedback cycles is always selected.
• Generality: While the closed-form solution is specific to the translated harmonic trap, the underlying recursion applies to arbitrary energy landscapes and observation channels, offering a framework for numerical optimization where analytic closure is not possible.

The authors note that these results are directly testable in existing feedback-trap experiments (e.g., colloidal information engines) and provide a rigorous design rule for the number of feedback cycles in such systems.