💬 NLP

Order Is Not Control

This paper argues that order-inducing interventions do not equate to control, proposing instead that genuine control requires a receiver-gated response law where local, state-dependent operators determine whether drives result in admitted movement, impedance, or overdrive, a framework validated through empirical evidence across biological systems and large language models.

Original authors: Gareth Seneque, Lap-Hang Ho, Nafise Erfanian Saeedi, Jeffrey Molendijk, Tim Elson

Published 2026-06-12

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Gareth Seneque, Lap-Hang Ho, Nafise Erfanian Saeedi, Jeffrey Molendijk, Tim Elson

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Idea: Pushing a Swing vs. Making It Swing

Imagine you are trying to get a child on a swing to go higher.

Order is just the child sitting on the swing.
Control is successfully making the swing go higher without breaking the chains or knocking the child off.

The authors argue that in both AI (like Large Language Models) and biology (like brains), we often confuse Order with Control. We see that a specific prompt, a biological signal, or a piece of code creates some structure or "order." But that doesn't mean we have Control.

True control only happens when you push the system, and it moves exactly where you want it to go, without causing damage, breaking down, or getting stuck.

The Core Concept: The "Receiver-Gated" Law

The paper introduces a strict rule for what counts as control. They call it a Receiver-Gated Response Law.

Think of the system (the AI or the brain) as a House.

The Drive: This is you knocking on the door (a prompt, a biological signal, or a command).
The Medium: This is the house itself (the AI model, the biological tissue, or the adapter software). Some houses have open doors; some have locked doors; some have traps.
The Receiver: This is the person inside the house who decides whether to let you in.
The Gate: The door.

The Rule: You only have "control" if:

You knock (apply a drive).
The person inside (the receiver) actually lets you in and moves to a specific room (the target outcome).
You didn't break the door, burn the house down, or get kicked out (no damage, no "null" responses, no "overdrive").

If you knock and the house ignores you, or if you knock and the house catches fire, you do not have control, even if you made a lot of noise (created "order").

The Experiments: Testing the House

The authors tested this idea across three different types of "houses" to see if the same rules apply.

1. The Biological Houses (Real Brains)

They looked at mice, worms, and fish. They applied electrical or chemical "knocks" to specific parts of their brains.

Result: They found that the brains have predictable "response laws." Sometimes a knock makes a muscle move (the door opens). Sometimes it does nothing (the door is locked). Sometimes it causes a spasm (the house catches fire).
Takeaway: Biology follows these rules, but it doesn't mean we can perfectly control a worm just by knowing which wire to touch. It depends entirely on the state of the worm at that moment.

2. The AI Houses (Language Models)

They tested AI models by giving them different prompts (knocks) and seeing what they wrote (the movement).

Result: They found that AI responses are predictable if you keep the conditions the same. If you ask a question in a specific way, the AI moves in a specific direction.
The Catch: If you push too hard (ask too many questions or use a very strong prompt), the AI might start hallucinating, refusing to answer, or writing nonsense. This is called "overdrive" or "sink routing."
Prediction: They could predict which way the AI would move (e.g., "it will likely become more helpful") about 73-84% of the time, but they couldn't guarantee the AI would do exactly what they wanted every single time without side effects.

3. The "Prepared Media" (The Renovation)

They tested "adapters" (small software updates that change how an AI thinks). They treated these like renovating the house.

Result: Changing the AI's training (renovating the house) changes how easily the door opens. Some renovations make the house more welcoming to certain requests; others make it more stubborn.
Takeaway: The renovation changes the susceptibility (how easy it is to get in), but it doesn't guarantee you can control the outcome. You still have to knock correctly.

The "Controller" Problem: Why We Can't Just "Steer" Yet

The paper concludes that while we can observe these laws and predict what might happen, we do not yet have a Controller.

Observation: We can see that "If I push here, the AI usually moves there."
Prediction: We can guess the outcome with high accuracy.
Control: To have control, we need a system that says: "The AI is currently in a 'fragile' state. If I push now, it will break. So, I will wait. If it is in a 'ready' state, I will push gently."

The authors found that while we can identify these "fragile" and "ready" states, we haven't built a system that can reliably choose the right action in real-time without making mistakes. We have the map, but we don't have the perfect driver yet.

Summary of What They Claim (and What They Don't)

What they proved:

Order (structure) is not the same as Control (getting the result you want).
Control requires a "receiver" to admit the action without causing damage.
This rule applies to both biological brains and AI.
We can predict how these systems will react to pushes, but the reaction depends heavily on the current state of the system and the environment.

What they did NOT prove:

They did not prove we can perfectly control AI or biology yet.
They did not prove that "principles" or "rules" written into a system automatically make it safe or aligned.
They did not claim that AI and biology are the same thing deep down; they just share the same rules of reaction.

The Final Metaphor: The Gardener

Think of the AI or the brain as a Garden.

Order is just the plants growing.
Control is making the plants grow exactly where you want, in the shape you want, without killing them.

The paper says: "We have figured out the laws of how water and sunlight affect these plants. We know that too much water drowns them, and too little starves them. We can predict how a plant will react to a specific amount of water. But we do not yet have a robot gardener that can look at every single plant, decide exactly how much water it needs right now, and water it perfectly without ever making a mistake."

We have the science of the garden; we are still working on the perfect gardener.

Technical Summary: Order Is Not Control – Driven-Dissipative Response Laws Across Artificial and Biological Systems

Problem Statement
The paper addresses a fundamental conflation in the study of adaptive systems (both biological and artificial): the confusion between the induction of order (structure, hidden state movement, or output regularity) and the establishment of control (reliable, bounded movement of a target outcome). Current alignment and interpretability research often identifies "order-inducing objects" (prompts, principles, activation edits, adapters) but fails to distinguish whether these objects produce receiver-admitted behavioral changes or merely induce internal reorganization that does not translate to the desired outcome. The core scientific problem is to separate order, response evidence, and local control into distinct empirical categories.

Methodology and Framework
The authors propose a receiver-gated response law framework, grounded in driven-dissipative and stochastic thermodynamic vocabularies. Control is defined not by the presence of an intervention, but by a specific condition: a finite intervention effort must move a target response class under a matched denominator while keeping damage, null/evasive responses, invalid formats, and unnecessary effort (sinks) bounded.

The Response Kernel: The empirical object is a denominator-conditioned stochastic response kernel, $P_\delta(dy | s, a)$ . Here, $\delta$ represents a tuple of conditions (task, medium, bath, receiver, basin rule, comparator). The kernel maps material state ( $s$ ) and action/drive ( $a$ ) to a distribution over response displacement, sink channels, and declared effort.
Denominator-Indexing: All measurements are strictly indexed by a denominator $\delta$ . A drive is only evidence of control if it moves the measured receiver under the same denominator. Changing the prompt, model state, decode settings, or biological protocol changes the denominator, rendering cross-denominator comparisons invalid for establishing control laws.
Evidence Hierarchy: The paper establishes a hierarchy of evidence:
1. Order: Induced structure or hidden movement.
2. Response Evidence: Reproducible movement in a declared response class under matched conditions.
3. Local Control: Finite effort moving a target while preserving side-effect bounds.
4. Controller Boundary: The limit where prospective action admission fails despite measurable opportunity.

Experimental Panels and Results
The study validates this framework across four complementary evidence domains:

Biological Response Operators (Physical Substrates):
- Substrates: Mouse Anterior Lateral Motor (ALM) cortex, C. elegans, and larval zebrafish.
- Method: Measured population response vectors and neural perturbation propagation under held-out protocols.
- Results: Demonstrated measurable response-operator structures with held-out sign accuracies of 0.716 (ALM), 0.597 (worm), and 0.577 (zebrafish).
- Limit: Supports physical response-operator structure but explicitly excludes coordinate identity or universal biological control.
Generated-Output Response Laws (LLMs):
- Method: Analyzed frozen completions under fixed prompts, decode baths, and rubrics. Tested visible semantic fields and finite action probes.
- Results: Showed that visible fields can move generated-output basins (e.g., target rate 1.000 vs. baseline 0.117) but exhibit non-monotone effort responses; stronger fields or higher budgets often route effort into damage or invalid sinks (overdrive).
- Key Finding: Response vectors are predictable (72.8–73.7% component-sign accuracy; 84.3–84.8% on nonzero components), but this does not equate to row-local target control.
Prepared-Medium Response (Adapters):
- Method: Compared frozen base models against constitution-conditioned LoRA adapters (editorial principles, NIST-style, null-random).
- Results: Adapters act as "prepared media" that reshape susceptibility (phase support, trajectory length, format admittance) rather than acting as autonomous controllers.
- Key Finding: Prepared media can expose "repair pockets" (e.g., reducing format invalidity) but do not establish universal adapter control or wording-as-causal alignment.
Predictive Operators and Controller Boundaries:
- Observer Prediction: Non-endpoint observers predicted held-out system-effect and target/oracle labels with high accuracy (93.6% and 91.7%, respectively).
- Local Admitted Control: Identified "clean pockets" where local actions successfully moved targets with bounded sinks (1,146 clean-positive cells).
- Stochastic Admission Failure: Despite measurable oracle headroom and retrospective opportunity, prospective held-out action admission failed (0/104 captures). Selected policies could not reliably choose the right action in the right state.

Key Contributions

Response-Chain Criterion: A formal operational definition separating order, response evidence, and control based on receiver-admitted movement and bounded sinks under a matched denominator.
Cross-Substrate Mapping: Demonstrates that biological, LLM, and adapter systems instantiate the same denominator-indexed response-law structure without requiring raw coordinate identity.
The Driven-Dissipative Account: Reframes alignment as an open-system regulation problem where drives act through prepared media and baths, producing admitted movement, impedance, or sinks.
Explicit Limits: Clearly delineates the boundary between measurable response operators and deployable controllers, showing that while local admitted control exists, prospective stochastic admission remains unsolved.

Significance and Claims
The paper argues that order is not control. It posits that alignment interventions (principles, prompts, adapters, steering) are only effective as control when they produce bounded, receiver-admitted movement under a specific denominator.

Modest Claims: The authors explicitly state they do not claim universal control, coordinate identity between biological and artificial systems, or that hidden/logit interventions are sufficient for behavioral control without receiver validation.
Significance: The work provides a "mesoscopic control level" account that makes the control problem explicit: estimate state, choose hold or an admitted action, and verify receiver movement with sink and effort channels bounded. It shifts the focus from "inducing order" to "measuring admitted response laws," identifying that while response vectors are predictable and local repair is possible, the deployment-grade controller (a policy that reliably selects actions across held-out states) is not yet established.

The paper concludes that alignment is a state-conditioned response reliability problem, requiring the separation of observer coordinates, candidate actuators, and validation gates, rather than a search for universal alignment scores or hidden-state sufficiency.