🤖 AI

Emergence of Physical Intelligence via Controllable Information Production

This paper introduces Controllable Information Production (CIP), a novel intrinsic motivation framework grounded in dynamical systems and optimal control that unifies physical intelligence with information production, enabling agents to master complex tasks like humanoid self-righting by driving systems toward the edge of controllable chaos.

Original authors: Tristan Shah, Stas Tiomkin

Published 2026-05-12

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Tristan Shah, Stas Tiomkin

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to teach a robot how to move, but you refuse to give it a cheat sheet or a reward system. You won't tell it, "Stand up and get a cookie," or "Walk forward and get a point." You want the robot to figure out how to be useful all on its own. This is the challenge of Intrinsic Motivation (IM): getting an agent to learn just by interacting with the world, without human instructions.

Previous attempts at this have been a bit like trying to teach a dog by guessing what it finds interesting. Researchers picked specific things to measure (like "how much does the robot move its arm?") and told the robot to maximize that. But this introduces human bias; the robot only learns what the human thought was interesting, not what is actually fundamental to the physics of the situation.

This paper introduces a new, cleaner way to do it called Controllable Information Production (CIP).

The Core Idea: The "Chaotic Tightrope"

Think of a robot as a tightrope walker.

Too stable: If the walker is standing on a wide, flat platform, they aren't doing much. They aren't learning, and they aren't really "in control" because nothing could go wrong.
Too chaotic: If the walker is on a wild, stormy ocean with no rope, they are just flailing. They are producing a lot of "information" (random movement), but they can't control it. They are just falling.
The Sweet Spot: The most interesting, intelligent behavior happens on the edge of chaos. This is like walking a tightrope in a strong wind. The walker is constantly fighting to stay upright. They are producing a lot of "information" (tiny adjustments, wobbles, corrections), but they are controlling it.

The authors argue that true physical intelligence isn't about being boringly stable or wildly chaotic. It's about finding that sweet spot where the system is unstable enough to be interesting, but controllable enough to be mastered.

What is "Controllable Information Production"?

In simple terms, CIP is a way for the robot to ask itself: "How much can I make the world change if I try to control it?"

The Old Way (The Biased Teacher): Previous methods asked the robot to maximize "diversity" or "curiosity." It's like a teacher saying, "Go find the most colorful things!" The robot might just spin in circles to see different colors, which isn't very useful.
The New Way (CIP): This method doesn't ask the robot to look for specific things. Instead, it measures the rate at which the robot creates new, unpredictable situations that it can still fix.

Imagine a child playing with a stack of blocks.

If the stack is already on the floor, the child can't do much (low information production).
If the child knocks the whole tower over instantly, they can't control the fall (high chaos, low control).
But if the child carefully balances the tower, making it wobble and adjusting their hands to keep it from falling, they are in the "CIP zone." They are generating a lot of complex, changing data (the wobbles), but they are the ones steering the outcome.

How It Works (The Magic Trick)

The paper connects this idea to Optimal Control, which is the math used to figure out the best way to drive a car or fly a plane.

The authors discovered a hidden link: The math that tells a robot how to stay stable (the "value function") also secretly tells you how much "controllable chaos" is happening.

They found a way to calculate this "chaos rate" without needing to guess which variables to measure.
They created a fast, stable calculator (an algorithm) that can run this math even on complex robots, something that was previously too hard to do.

The Results: Robots That Stand Up

The researchers tested this on several robot simulations, including:

Cart Poles: A stick on a moving cart.
Double and Triple Pendulums: Sticks hanging from sticks, which are notoriously hard to balance because they swing wildly.
A Gibbon: A robot that looks like a human hanging from a bar, trying to pull itself up to a standing position.

The Outcome:
Without any human rewards or instructions, the robots using CIP figured out how to:

Swing up and balance the sticks.
Pull themselves from a hanging position to a standing position (self-righting).

Other methods (like "curiosity" or "diversity" seekers) mostly failed. They got stuck in loops or couldn't figure out how to stabilize the upright position. The CIP robot, however, naturally gravitated toward the upright position because that is where the "controllable information production" is highest. It realized that standing up is the most "interesting" state to control.

Why This Matters

This paper suggests a new rule for how intelligent systems should learn: Drive systems toward the edge of controllable chaos.

Instead of telling a robot what to do, you give it a compass that points toward "interesting, controllable instability." The robot then discovers useful skills (like standing up or balancing) on its own, because that is where the math says the most interesting control is happening.

In a nutshell: The paper gives robots a new internal compass. Instead of looking for rewards, they look for the "Goldilocks zone" of physics—where things are unstable enough to be challenging, but controllable enough to be mastered. This allows them to learn complex physical skills, like standing up, without a human ever telling them to do it.

Technical Summary: Emergence of Physical Intelligence via Controllable Information Production

Problem Statement

Current approaches to Intrinsic Motivation (IM) in robot learning face three fundamental structural deficiencies. First, they predominantly rely on mutual information between designer-chosen random variables, serving as proxies for system-level phenomena like complexity or chaos rather than measuring the phenomena directly. Second, the necessity for a designer to explicitly specify these variables introduces bias, implicitly constraining how an agent perceives and acts. Third, despite being used to drive control, these objectives lack a principled connection to Optimal Control (OC) theory, limiting interpretability and the development of efficient control methods. Consequently, existing IM methods often fail to induce genuinely novel, task-relevant physical behaviors, such as self-righting, without external reward signals.

Methodology: Controllable Information Production (CIP)

The authors introduce Controllable Information Production (CIP), a new IM paradigm grounded directly in the invariant properties of dynamical systems and Optimal Control theory.

Theoretical Foundation

CIP is defined as the rate at which a controlled system produces information along its trajectories, formally identified as the Kolmogorov-Sinai entropy (KSE) induced by the agent's actions.

Open-Loop vs. Closed-Loop: The framework distinguishes between the information production rate under feedback control (closed-loop KSE, $h_{ks}(f^{cl})$ ) and the rate if perturbations were left unregulated (open-loop KSE, $h_{ks}(f^{ol})$ ).
Derivation from Optimal Control: The authors demonstrate that CIP emerges naturally from the Hessian of the optimal value function in OC. By decomposing the optimal policy into extrinsic (goal-directed drift) and intrinsic (perturbation-regulating feedback) components, they show that the value Hessian encodes the growth of infinitesimal perturbations.
Key Insight: The open-loop KSE ( $h_{ks}(f^{ol})$ ), derived from the value Hessian, serves as the CIP objective. The authors prove that CIP provides an upper bound on the closed-loop KSE ( $CIP \ge h_{ks}(f^{cl})$ ), meaning maximizing CIP drives the system toward regions of high "controllable chaos"—states that are unstable in open-loop but stabilizable via feedback.

Efficient Estimation

Calculating KSE in high-dimensional systems is historically challenging. The paper proposes a numerically stable, scalable estimator derived from the Discrete Algebraic Riccati Equation (DARE).

Instead of propagating the Hessian matrix directly (which grows exponentially), the method propagates the log-determinant of the inverse Hessian in a log-domain recursion.
This approach bypasses the need for QR decomposition, which is standard in Lyapunov spectrum estimation but computationally expensive and unstable in high dimensions.
The estimator converges to known KSE values in chaotic systems (e.g., Lorenz attractors) faster than baseline QR methods.

Implementation

In experiments, CIP is implemented using a random-sampling-based Model Predictive Control (MPC) scheme. The agent optimizes a control sequence over a finite horizon to maximize the estimated CIP, subject to a control penalty. No external rewards or demonstrations are used.

Key Contributions

New IM Paradigm: The introduction of CIP, which shifts the objective from promoting novelty/diversity to identifying and stabilizing salient regions of state space (the "edge of chaos") based on dynamical invariants.
Theoretical Grounding: A rigorous connection between IM and Optimal Control, showing that CIP is encoded in the curvature of the optimal value landscape. This provides a principled justification for IM that does not rely on heuristic postulates.
Scalable Estimator: The development of an efficient, numerically stable recursive estimator for KSE that operates directly on the system's Jacobians, overcoming previous limitations in high-dimensional applications.
Empirical Validation: Demonstration that CIP outperforms state-of-the-art IM methods (including Empowerment, DIAYN, DADS, SMM, and ICM) on standard benchmarks, uniquely enabling complex physical skills like humanoid self-righting without external rewards.

Experimental Results

The authors evaluated CIP on four benchmark systems in Mujoco-MJX: Cart Pole, Double Pendulum, Triple Pendulum, and a planar humanoid (Gibbon).

Self-Righting Performance: CIP consistently drove agents to the fully upright position (normalized height $\approx 0.93–0.99$ ) across all environments.
Comparison with Baselines:
- Skill-based IM (DIAYN, DADS, SMM): These methods failed to stabilize the upright position, often settling for oscillatory or suboptimal stable poses. The authors attribute this to the "selective pressure" for state-space diversity, which favors easier-to-maintain states over the difficult upright configuration.
- Curiosity-based IM (ICM): These methods failed to stabilize, often fixating on unpredictable high-velocity motion or ceasing to be "surprised" as predictors improved.
- Empowerment: While successful in the Cart Pole, Empowerment only partially succeeded in the Double Pendulum and failed to stabilize the Triple Pendulum or Gibbon, falling to suboptimal states.
Entropy Correlation: The estimated CIP values increased as the agents moved from hanging (low entropy) to upright (high entropy) configurations, confirming that the upright state corresponds to high open-loop information production.

Significance and Claims

The paper posits that physical intelligence emerges from driving systems toward the edge of controllable chaos. By maximizing the information production an agent can control, CIP provides a principled mechanism for discovering useful behaviors without human bias.

Theoretical Impact: The work unifies IM and OC, formalizing physical intelligence as the control of information production. It reveals a structural connection between the value function and Kolmogorov-Sinai entropy.
Practical Impact: CIP enables agents to learn complex, energy-efficient behaviors (like self-righting) that are prerequisites for locomotion, which prior IM methods failed to achieve autonomously.
Limitations: The authors acknowledge that the current implementation relies on differentiable simulators to access Jacobians and uses a simple MPC controller, which may limit scalability to very high-dimensional real-world systems. Future work aims to integrate CIP with learned dynamical models and state-of-the-art RL policies.

The authors conclude that CIP represents a general organizing principle for physical intelligence, moving beyond heuristic objectives to a framework derived from the fundamental mathematics of control and dynamical systems.