💻 computer science

Control-aware training of physical neural networks for closed-loop regulation

This paper demonstrates that optimizing physical neural networks with control-aware objectives, which prioritize preserving candidate-action rankings over static regression accuracy, significantly enhances closed-loop regulation performance and robustness against perturbations in tasks like homeostatic regulation and Cart-Pole stabilization.

Original authors: Yuzhan Zhang

Published 2026-08-25

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Yuzhan Zhang

Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

In the world of modern computing, there is a growing movement to build machines that think not just with silicon chips, but with the raw, physical laws of the universe itself. These are called physical neural networks. Instead of simulating a brain on a standard computer, these systems use real, controllable physical processes—like light passing through glass or the vibration of tiny mechanical oscillators—to perform calculations. They are fast and efficient, but they are also messy. The physical world is full of small, unavoidable imperfections: temperature shifts, slight manufacturing differences, and random noise. When a computer program runs on a standard processor, these tiny errors usually don't matter much. But when a machine is used to control something in real time, like balancing a pole or regulating a chemical reaction, even a tiny mistake in calculation can lead to a wrong decision. If a controller misjudges the best move by a fraction, it might choose a slightly worse action. In a feedback loop, that small error can compound, sending the system down a path that eventually leads to failure. The central question for scientists is whether these physical machines can be trained to ignore their own imperfections and make the right choices, even when the numbers they calculate are slightly off.

A researcher named Yuzhan Zhang tackled this problem by asking a different question about how these machines should be taught. Traditionally, engineers train these physical systems by trying to make their output numbers match a perfect target as closely as possible, much like a student trying to get every answer on a math test exactly right. Zhang's work suggests that for machines that control physical systems, getting the numbers exactly right is not the most important goal. Instead, the most important goal is to ensure the machine picks the right order of actions. If a controller has to choose between moving left or right, it does not need to know the exact value of "left" and "right" to the last decimal point. It only needs to know that "left" is better than "right" by enough of a margin to be sure. The study shows that by training the physical network to protect this ranking of choices, rather than just minimizing numerical errors, the system becomes much more robust when things go wrong in the real world.

To test this idea, the researcher built two different types of physical computing systems in a simulation. One system used a model of light interfering with itself, similar to how waves in a pond overlap, while the other used a network of twenty-four tiny, connected oscillators that swing back and forth like pendulums. These systems were tasked with a job called homeostatic regulation, which is essentially keeping a set of internal variables, such as energy and temperature, within a safe, healthy range. The systems had to constantly decide whether to charge up, cool down, or repair themselves to avoid a dangerous state. The researcher trained two versions of the controller for each system. The first version was trained in the traditional way, trying to match the perfect numbers as closely as possible. The second version was trained with a new method that focused on keeping the ranking of the best actions correct, even if the numbers themselves were a bit fuzzy.

The results were striking. When the systems were tested under normal, perfect conditions, both controllers performed almost identically. They made the same decisions and kept the internal variables in the safe zone with nearly the same efficiency. However, the true test came when the researcher introduced realistic disturbances, such as random noise in the sensors or slight changes in the physical properties of the machine. Under these stressed conditions, the traditional controller began to falter. It started making choices that led to higher costs and a greater risk of failure. The controller trained with the new, ranking-focused method, however, held its ground. In the light-based system, this new approach reduced the cost of keeping the system alive by nearly eighteen percent when facing moderate to strong disturbances. In the oscillator system, the improvement was even more dramatic, cutting the cost by nearly twenty-nine percent. Crucially, this improvement happened even though the new controller was not better at predicting the exact numbers; in fact, in some tests, its raw numerical predictions were slightly worse than the traditional model. The advantage came entirely from its ability to maintain the correct order of choices when the environment was noisy.

To ensure these findings were not just a fluke of one specific task, the researcher also tested the method on a classic control problem known as the Cart-Pole, where a machine must balance a pole on a moving cart. This is a standard test for control systems, unrelated to the homeostatic task. Again, the two controllers were matched so that their ability to predict numbers was nearly identical. When the system was subjected to strong disturbances, the controller trained to protect the ranking of actions reduced the rate of failure by more than half compared to the traditional controller. It kept the pole balanced for significantly longer periods, surviving hundreds of steps where the other model failed quickly. The study also looked at whether the new controller was simply being more cautious, perhaps doing less work to avoid risk. By carefully comparing the two systems when they were forced to do the exact same amount of work, the researcher confirmed that the improved performance was not due to laziness or reduced effort. The new controller was genuinely better at making the right decision in the moment, which prevented the system from drifting into a dangerous state.

The research also included a check against real-world data from other experiments to see if the level of difficulty used in the simulations was realistic. The study found that the disturbances used in the tests were comparable to the errors seen in actual physical computing hardware reported in other scientific papers. This suggests that the improvements found in the simulation are relevant to real machines that might be built in the future. The work does not claim to have solved all problems with physical computing, and it acknowledges that in one specific test on the oscillator system, the improvement was small and did not meet a strict threshold for success. However, the overall pattern is clear: when a physical machine is embedded in a feedback loop, training it to care about the consequences of its choices is more effective than training it to be a perfect calculator. The study concludes that for closed-loop control, the structure of the decision matters more than the precision of the number. By teaching these physical networks to value the correct ordering of actions, engineers can build systems that are far more resilient to the inevitable messiness of the physical world.

Technical Summary: Control-Aware Training of Physical Neural Networks for Closed-Loop Regulation

Problem Statement

Physical Neural Networks (PNNs) are computational systems that exploit controllable physical dynamics (e.g., photonic, electronic, or mechanical) as trainable transformations. While recent advances have improved methods for training the physical processor itself to approximate input-output maps, a critical gap remains when these processors are embedded in closed-loop feedback control.

In open-loop settings, approximation error is typically measured by global regression metrics like Mean Squared Error (MSE). However, in closed-loop regulation, small approximation errors can have disproportionate consequences if they alter the ordering of candidate actions near decision boundaries. Two estimators with similar global accuracy can induce divergent trajectories if their errors are distributed differently near action-ranking boundaries. Conversely, larger numerical errors may be behaviorally inconsequential if the margin between the best and second-best actions is sufficiently large. The paper argues that static regression accuracy is an incomplete deployment metric for physics-constrained estimators used in sequential decision-making.

Methodology

Theoretical Framework

The author derives theoretical bounds linking approximation error to decision consequences:

Action-Margin Bounds: Propositions 1–3 establish that one-step decision regret is bounded by $2\varepsilon$ (where $\varepsilon$ is the uniform approximation error). If the action gap (difference between the best and second-best action scores) exceeds $2\varepsilon$ , the optimal action is preserved.
Trajectory Divergence: Finite-horizon analysis links trajectory divergence to the accumulation of action-gap risks, showing that the probability of divergence depends on the sub-Gaussian tail bounds of score errors relative to action gaps.

Control-Aware Training Objective

To address these theoretical insights, the paper proposes a composite training objective ( $L$ ) that goes beyond standard regression:
$L = \lambda_{field}L_{field} + \lambda_{rank}L_{rank} + \lambda_{boundary}L_{boundary} + \lambda_{robust}L_{robust}$

$L_{field}$ : Fits the reference viability function (standard regression).
$L_{rank}$ : Penalizes the reversal of candidate-action rankings (preserving the order of actions).
$L_{boundary}$ : Emphasizes states near viability transitions.
$L_{robust}$ : Averages performance over physically motivated perturbations (input noise, parameter drift, readout noise).

Experimental Design

The study employs a rigorous "frozen" protocol to ensure reproducibility and avoid overfitting to held-out data. Key features include:

Static Matching: Models are selected based on strict static metrics (e.g., $R^2 \ge 0.99$ , RMSE ratios) before evaluating closed-loop performance. This ensures that any observed behavioral differences are not due to one model simply having better static accuracy.
Substrates: Experiments were conducted on two independently implemented physical substrates:
1. Interferometric: A coherent-interference architecture with 64 paths and 16 detectors.
2. Nonlinear Oscillator: A network of 24 damped, nearest-neighbor coupled Duffing oscillators.
Tasks:
1. Homeostatic Regulation: An agent regulates internal state variables (energy, temperature, risk) against a viability cost.
2. Cart–Pole Stabilization: A canonical control problem involving balancing a pole on a cart.
Perturbations: Evaluations included "moderate" and "strong" perturbation conditions combining input/readout noise with phase-offset and coupling drift, calibrated against reported errors in real-world PNN deployments.

Key Results

1. Static Accuracy vs. Closed-Loop Behavior (E-010C1)

Two models with nearly identical static accuracy ( $R^2 \approx 0.9917$ , global RMSE ratio $\approx 1.005$ ) were compared.

Result: The control-aware model reduced the mean viability cost by 14.79% compared to the MSE-only baseline.
Observation: The control-aware model had slightly worse pointwise RMSE on candidate states but significantly better action agreement with the reference controller. This demonstrates that static accuracy does not guarantee closed-loop performance.

2. Robustness to Physical Perturbations (E-020C1 & E-031C1)

Interferometric Substrate: Under moderate-to-strong perturbations, robust control-aware training reduced viability cost by 17.86% compared to boundary-aware training. This improvement persisted even after matching task utility (E-050C1), ruling out the hypothesis that the gain was due to performing less work.
Oscillator Substrate: In a robustness test (E-031C1), control-aware training reduced viability cost by 28.63% under perturbations, despite having slightly lower unperturbed static accuracy ( $R^2$ ) than the boundary-aware baseline.
Nominal Replication (E-030C1): A direct replication of the static-matching design on the oscillator substrate showed a stable direction of improvement (4.14% reduction) but failed to meet the pre-defined 5% practical-effect threshold.

3. Mechanism Validation (E-051C1)

Using a common pool of counterfactual states generated by an exact reference controller:

Result: Control-aware models reduced exact one-step decision regret by 49.2% (interferometric) and 18.6% (oscillator).
Significance: This confirms that the improvement stems from preserving the correct action ordering (consequence-awareness) rather than just reducing global error.

4. Generalization to Cart–Pole (E-060C1)

The findings were generalized to the standard Cart–Pole task:

Result: Two estimators with nearly identical global static accuracy (RMSE ratio 0.999) were tested. The control-aware model had worse candidate-state pointwise RMSE but improved failure-padded stabilization loss by 54.63% under perturbations.
Outcome: Survival steps increased significantly (e.g., from 103 to 248 under strong perturbation), and success rates improved from 0% to 16.4%.

5. External Audit (E-040)

A descriptive audit compared the synthetic stress regimes used in the study against reported errors in published experimental PNNs (MRR and free-space systems). The synthetic "moderate" and "strong" perturbations induced degradation magnitudes comparable to real-world deployment errors, validating the relevance of the stress tests.

Significance and Claims

The paper claims to identify decision-consequence structure as a more relevant training target for closed-loop physical computing than static regression accuracy alone.

Core Contribution: The study provides a training and evaluation principle for closed-loop PNN deployment. It demonstrates that protecting candidate-action rankings (via control-aware objectives) yields more robust regulation under physical perturbations, even when static metrics are matched or slightly worse.
Modesty: The author explicitly states that this is a computational training principle tested on physics-constrained models, not a claim of universality across all physical systems or a theory of artificial consciousness.
Limitations: The study acknowledges that the nominal viability gain in the homeostatic task (E-010) carried a task-utility trade-off that did not survive a task-matched audit. Additionally, the cross-substrate nominal replication (E-030) failed to meet the strict 5% effect threshold.
Scope: The work isolates algorithmic effects from substrate constraints by using simulated physics-constrained models. It does not claim device-level efficacy on third-party hardware, noting that a future hardware study is required to establish that.

In summary, the paper argues that for PNNs in feedback loops, how errors are distributed (specifically regarding action margins) matters more than how large the errors are globally. Training objectives that explicitly account for these margins can significantly improve robustness and stability in the face of physical non-idealities.