🔢 mathematics

The Semantic Least-Energy Principle: A Hypothesis for Intelligence

This paper proposes the Semantic Least-Energy Principle (SLEP), a variational hypothesis suggesting that intelligent systems organize semantic representations by maximizing utility while minimizing semantic, predictive, and computational energy, thereby unifying abstraction, reasoning, and communication within a single mathematical framework.

Original authors: Jie Zhang, Haoyuan Zhu, James Jinheng Zhang, Haonan Hu

Published 2026-07-28

📖 8 min read🧠 Deep dive

CC BY 4.0

Original authors: Jie Zhang, Haoyuan Zhu, James Jinheng Zhang, Haonan Hu

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to organize a massive, chaotic library. You have millions of books, but they are all jumbled up, covered in dust, and written in thousands of different languages. If you want to find a specific story quickly, you can't just look at every single book. You need a system. Maybe you group them by genre, then by author, then by color. This is what scientists call "compression": taking a huge amount of messy information and squeezing it into a smaller, cleaner format that still keeps the important parts.

For a long time, scientists have known that both human brains and computer brains (Artificial Intelligence) do this. They take the noisy, complicated world around them—like the sound of a dog barking or the pixels of a photo—and turn it into a neat, internal "map" or "idea" that helps them make decisions. We know that they do this, and we have some math to describe how they send messages or learn from mistakes. But we don't really know why they organize these internal maps the way they do. Is there a single, golden rule that explains why a smart system chooses one way to think over another? It's like knowing a car has an engine, but not knowing the law of physics that makes the wheels turn.

This is the big question a new paper by Jie Zhang and his team is trying to answer. They are looking at the intersection of computer science, math, and how our brains work. They want to find a "first principle"—a fundamental rule of nature—that explains how intelligence organizes its thoughts. They aren't just looking at how to make a better chatbot; they are asking, "What is the deep, underlying logic that makes any intelligent system smart?"

The Paper's Big Idea: The Semantic Energy Principle

The authors propose a new idea called the Semantic Least-Energy Principle (SLEP). Think of it like this: Imagine your brain is a hiker trying to get from the bottom of a mountain to the top. The "mountain" is all the possible ideas and concepts you could think about. The "top" is the best answer to a problem.

Usually, we think of intelligence as just trying to get to the top as fast as possible. But this paper suggests that smart systems are actually trying to do two things at once:

Get the right answer (keep the "semantic utility" high).
Use as little energy as possible to get there.

The authors call this "semantic energy." It's not the electricity in a computer or the calories in your brain; it's a mathematical way of measuring how "expensive" or "clunky" a thought is. A thought that is confusing, redundant, or requires a huge mental leap has high energy. A thought that is simple, clear, and efficient has low energy.

The Main Finding: The paper suggests that intelligent systems (like AI or humans) naturally evolve to find the "path of least resistance" on this mountain of ideas. They organize their internal thoughts so that the most useful concepts are the easiest to reach. It's like the water in a river: it doesn't just flow randomly; it finds the path of least energy to get to the ocean. The authors suggest that intelligence does the same thing with ideas.

What They Are Arguing Against

The paper is careful to say what this idea is not. It argues against the idea that intelligence is just about memorizing facts or minimizing errors in a simple way.

It's not just about predicting the future perfectly (like guessing the next word in a sentence).
It's not just about compressing data to save space (like a ZIP file).
It's not about reconstructing an image exactly as it was.

Instead, the authors suggest that all these things are just different sides of the same coin. They argue that when a system learns, it isn't just trying to be "right"; it is trying to become "efficient." It wants to strip away the unnecessary noise and keep only the core meaning, because that is the path of least energy.

The Five Big Guesses (Hypotheses)

The paper doesn't claim to have proven this with a magic wand. Instead, they offer five "hypotheses" (educated guesses) that build a new way of looking at intelligence. They treat these like a map for future explorers to test.

1. The Low-Energy Dream
The first guess is that as a smart system learns, it naturally drifts toward "low-energy" states. Imagine a messy room where you keep throwing things away until only the essentials remain. The paper suggests that as an AI learns, its internal "map" of the world gets simpler and more efficient, while still keeping all the important information. They predict that if you measure the "energy" of a learning AI, it should go down over time, even after it has already learned how to do the task well.

2. The Straight-Line Thought
The second guess is about how thoughts move. In physics, if you throw a ball, it follows a curve called a "geodesic" (the shortest path between two points on a curved surface). The authors suggest that when a smart system reasons or plans, its thoughts don't jump around randomly. Instead, they follow smooth, efficient paths through the "landscape" of ideas. If you could watch an AI thinking, its thoughts would look like a smooth slide down a hill, not a jagged, bumpy ride.

3. The Shape of Meaning
The third guess is that ideas have a shape. Just as a city has streets and distances between buildings, the authors suggest that "concepts" have a geometry. Two ideas that are similar in meaning (like "cat" and "kitten") should be close together in this mental map, regardless of whether you see them as a picture or hear them as a sound. They call this "Semantic Geometry." They predict that if you map out an AI's thoughts, you'll see that similar ideas naturally cluster together, forming a smooth, curved surface.

4. The Heat of Thinking
The fourth guess is a bit wild: they suggest that groups of thoughts behave like gas molecules in a box, which is a concept from "thermodynamics" (the study of heat and energy). In a hot gas, molecules move around wildly. In a cold gas, they settle down. The authors propose that "semantic temperature" is a measure of how uncertain or chaotic a system's thoughts are. When an AI is confused, its "semantic temperature" is high. When it has a solid, stable understanding of the world, its "temperature" drops, and its thoughts settle into a calm, predictable pattern.

5. The Grand Unification
The final guess is that all these ideas—learning, reasoning, planning, and talking—are actually the same thing happening in different ways. The authors suggest that the "Semantic Least-Energy Principle" is the master key. Whether it's a robot learning to walk, a human learning to speak, or a computer learning to write poetry, they are all following the same rule: Keep the useful stuff, drop the rest, and take the easiest path.

How Sure Are They?

It is very important to understand that the authors are not saying they have solved the mystery of intelligence. They are very clear about this. They call their work a "hypothesis" and a "framework," not a law of physics.

They haven't proven it yet. They haven't run a final experiment that says, "Aha! We were right!"
They have built a math playground. They created a set of equations and rules that could explain how intelligence works.
They are inviting others to test it. The paper ends with a list of specific things scientists should look for to see if the idea is true. For example, they say, "If our idea is right, then when you train an AI, its 'energy' should drop in a specific way." If scientists check this and it doesn't happen, then the idea is wrong, and they will know to throw it out.

The authors are essentially saying, "Here is a new map. It looks promising, and it connects a lot of different dots we've seen before. But we need you to go out into the world and check if the terrain actually matches the map."

Why This Matters

Why should a curious teenager care about "semantic energy" or "geodesics"? Because if this idea is right, it changes how we build the future.

Right now, we build AI by trial and error. We try a new design, see if it works, and if it doesn't, we try something else. If the Semantic Least-Energy Principle is true, we could stop guessing. We could design AI systems that are built to follow the path of least energy from the start. This could lead to smarter, faster, and more efficient computers that don't need to be as huge or power-hungry as the ones we have today.

It also gives us a new way to understand our own brains. Maybe the reason we get tired after thinking hard is that we are burning through "semantic energy." Maybe the reason we have "aha!" moments is that we finally found the low-energy path to the answer.

The paper doesn't give us all the answers, but it gives us a new question to ask: What if intelligence is just the universe's way of finding the easiest way to understand itself? That's a pretty cool idea to think about.

Technical Summary: The Semantic Least-Energy Principle (SLEP)

Problem Statement
Despite significant advances in artificial intelligence (deep learning, large language models, world models) and cognitive neuroscience (predictive coding, efficient coding), there is no generally accepted first-principle hypothesis explaining why intelligent systems organize latent semantic states as they do. Existing frameworks, such as Information Theory, the Information Bottleneck, and the Free Energy Principle, provide powerful tools for understanding communication, compression, and prediction but do not explicitly address the emergence, organization, and evolution of semantic intelligence itself. Specifically, current optimization objectives (likelihood maximization, prediction accuracy, reconstruction fidelity) do not explicitly model the trade-off between preserving behaviorally relevant semantic structure and minimizing the costs associated with semantic, predictive, and computational energy.

Methodology
The paper proposes the Semantic Least-Energy Principle (SLEP) as a variational hypothesis. The methodology formalizes semantic intelligence as a constrained optimization problem governed by a Semantic Action Functional. The framework is built upon three complementary components:

Formal Semantic Variational Framework:
- Semantic Action Functional ( $J[z]$ ): Defined as $J[z] = \int_{t_0}^{t_1} (U_S(z) - \lambda E_S(z, \dot{z})) dt$ , where $U_S$ is semantic utility, $E_S$ is semantic energy, and $\lambda$ is a trade-off parameter.
- Stationary Solutions: Applying the calculus of variations yields semantic Euler–Lagrange equations, suggesting that intelligent cognitive processes evolve along approximately stationary-action trajectories on a latent semantic manifold.
- Axioms: The framework rests on four axioms: (1) Learning dynamics follow utility-constrained free-energy descent; (2) Semantic distinguishability is measured by the Fisher metric; (3) Inference dynamics follow an overdamped Langevin equation on the manifold; (4) Goal-directed planning follows a variational principle minimizing action subject to utility constraints.
Semantic Manifold Geometry:
- The latent semantic space is equipped with a Riemannian geometry defined by the Fisher–Rao semantic metric ( $g_{ij}$ ), derived from the learned decoder distribution $p_\psi(x|z)$ .
- This metric defines semantic distance, geodesics, and curvature, positing that semantic reasoning follows geodesic paths rather than arbitrary feature transformations.
Semantic Thermodynamics:
- The framework extends to statistical ensembles of semantic states, introducing concepts of semantic entropy, semantic temperature ( $T$ ), and semantic free energy ( $F_S$ ).
- It derives a Boltzmann-like equilibrium distribution ( $q^*(z) \propto e^{-V(z)/T}$ ), where $V(z)$ is the semantic potential (expected predictive cost).

Key Contributions and Theoretical Results
The paper does not present new empirical data but offers a mathematically coherent theoretical framework that unifies existing theories. Key theoretical contributions include:

Unification of Existing Frameworks (Theorem 9): The authors demonstrate that the objective functionals of the Information Bottleneck, Variational Autoencoders (VAEs), Free Energy Principle (FEP), Maximum-Entropy Reinforcement Learning, Natural Gradient learning, and World Models can all be derived as special cases or correspondences within the SLEP framework under specific substitutions.
Derivation of Semantic Dynamics:
- Theorem 5: Establishes that the most probable paths of the inference dynamics obey an Onsager–Machlup variational principle, leading to a second-order semantic trajectory equation.
- Theorem 6 & Corollary 6.1: Show that in regions of small potential gradients and low noise, inference trajectories concentrate on geodesics, and planning trajectories (Axiom 4) align with inference trajectories (Axiom 3).
- Theorem 8: Proves that under the proposed learning axiom, the projected descent flow ensures the learning objective (Free Energy + Computational Cost) is non-increasing.
Multimodal Alignment (Theorem 10): Demonstrates that enforcing a cross-modal objective leads to a bound on the geodesic distortion between latent representations of different modalities, providing a theoretical basis for semantic alignment.
Operational Definitions: The paper rigorously defines semantic utility (prediction, control, communication, reasoning), semantic energy (predictive cost), and semantic temperature (a macroscopic descriptor of uncertainty).

Predictions and Testable Hypotheses
The paper outlines five specific, falsifiable predictions to guide future empirical studies in AI and neuroscience:

Energy Decrease: Semantic energy (predictive cost) should decrease during learning, continuing even after utility (task performance) plateaus.
Trajectory Concentration: Learned semantic trajectories should concentrate on low-action Onsager–Machlup paths, with deviations correlating positively with external novelty.
Geodesic Behavior: In regions of low semantic potential gradient, deliberation trajectories should approximate geodesics with minimal deviation.
Boltzmann Organization: There should be an affine relationship between self-information ( $I$ ) and semantic potential ( $V$ ) across states, allowing for the estimation of a semantic temperature.
Alignment Scaling: Cross-modal geodesic distortion should decrease monotonically as the weight of the cross-modal objective increases.

Significance and Claims
The authors explicitly state that SLEP is not a claim of an established law of intelligence. Instead, its significance lies in providing a mathematically coherent, conceptually unified, and experimentally falsifiable framework that connects disparate theories of intelligence (from Information Theory to Deep Learning).

Conceptual Unification: It offers a first-principle perspective on why semantic abstraction emerges, suggesting it is a consequence of an underlying variational optimization process that balances utility against energy.
Generative Hypothesis: By framing perception, reasoning, planning, and communication as manifestations of a single variational principle, it generates precise predictions for future empirical validation.
Scientific Rigor: The paper emphasizes that if these predictions fail across broad classes of intelligent systems, the hypothesis will require refinement or rejection, thereby advancing understanding through the process of scientific falsification.

The work positions itself as a foundational step toward a first-principle theory of semantic intelligence, bridging the gap between how systems are currently optimized and the fundamental principles governing the organization of latent semantic states.