Integrating Out, Twice:The Open-System Case That Neural-Network Ensemble Theory Is Missing

This paper establishes a theoretical framework comparing closed-system neural network ensembles with open-system analogs from nuclear reaction theory, ultimately concluding that the latter's distinctive non-Hermitian dynamics are structurally absent in mainstream learning due to the lack of continuous spectra and wave-like behavior, thereby locating the true source of operational uncertainty within the closed-system correspondence.

The Gist

The Big Idea: Two Ways to Ignore Things

Imagine you are trying to understand a complex system, like a crowded room or a neural network (a type of AI). Sometimes, you can't track every single person or every single number. You have to decide to ignore part of the system to focus on the part you care about.

In physics and math, this act of "ignoring" or "integrating out" a part of a system is a standard move. The author, Jin Lei, argues that there are two very different ways to do this, and while AI researchers mostly use one, nuclear physicists have mastered the other.

1. The "Closed" Way (What AI Does)

The Analogy: Imagine you are taking a photo of a group of friends, but you decide to blur out the background.

• What happens: You lose the details of the background, but the photo of your friends remains perfectly clear and "whole." The blur doesn't steal any light or energy from your friends; it just removes the background data.
• In AI: When AI researchers average out random numbers (parameters) in a neural network, they get a "closed" result. The math stays simple, real, and symmetrical. It's a lossless summary. Nothing "escapes."

2. The "Open" Way (What Nuclear Physics Does)

The Analogy: Imagine you are in a room with a door that is slightly ajar. You are trying to track the air pressure inside the room.

• What happens: Air leaks out through the door. If you try to describe the air only inside the room, your description must account for the fact that air is leaving. The math becomes "leaky" and complex. You have to keep a strict ledger (a receipt) of exactly how much air escaped and where it went.
• In Nuclear Physics: This is called the Optical Model. When a nucleus interacts with particles, some particles escape into the "continuum" (the rest of the universe). The math describing the nucleus becomes "non-Hermitian" (a fancy way of saying it's complex and leaky). Crucially, the math includes a Flux Ledger: an exact accounting of the probability that left the system.

The Paper's Main Claim

The author says: "AI is only doing the 'Closed' version. It is missing the 'Open' version."

AI researchers have a great dictionary for translating between their "Closed" math and nuclear physics. For example:

• The Neural Tangent Kernel (how AI learns) is the same as the Fisher Sensitivity Kernel (how sensitive a nuclear model is to changes).
• Infinite-width AI is the same as a Gaussian Process (a standard statistical tool).

However, the author argues that AI is blind to the "Open" side. AI treats any information it discards (like ignoring a word in a sentence or cutting off a part of a network) as a simple mistake or approximation error. It doesn't treat it as physical loss that needs to be tracked and conserved.

The "Flux Ledger"

In nuclear physics, when particles escape, the theory doesn't just say, "Oops, we lost some." It says, "We lost exactly 0.5 units of probability to Channel A and 0.2 to Channel B, and here is the math proving it."

The author tried to build this "Flux Ledger" for AI. He asked: If we treat an AI's "ignored" parts as a leaky door, can we track the lost probability?

The Surprising Result (The "Negative" Finding)

The author ran tests to see if this "Open" math worked for real AI models (like attention mechanisms in Large Language Models or routers that pick which experts to use).

The Result: It mostly failed.

• Why? For the "Open" math to work, the part you ignore needs to be like an infinite ocean where waves can travel forever (a continuous spectrum).
• The Problem: AI models are usually finite and "dissipative" (they relax and settle down). They don't have that "infinite ocean" quality.
• The Consequence: When the author tried to force the "Open" math onto AI, the "Flux Ledger" either didn't exist, or the "loss" was just an artifact of how he cut the data, not a real physical property.

The "Hallucination" Twist

The author also looked at a popular idea: Can this "leakage" math detect when an AI is hallucinating (making things up)?

The Answer: No.

• The Reason: When an AI hallucinates confidently, it is actually very "closed." It is committing strongly to a wrong answer. The "leakage" (uncertainty) is low because the model is sure of itself.
• The Real Uncertainty: The uncertainty that matters (Epistemic uncertainty—whether the model knows the answer) lives in the "Closed" part of the math (the variance of the ensemble), not the "Open" part.

Summary

• The Map: The paper draws a map showing that AI and Nuclear Physics share the same algebra for "ignoring" things.
• The Gap: AI only uses the "Closed" (lossless) version. Nuclear Physics has a fully developed theory for the "Open" (leaky) version, including a strict accounting of what is lost.
• The Test: The author tried to bring the "Open" theory into AI.
• The Verdict: It didn't work well. Real AI models are too finite and "relaxational" to support the complex, wave-like "Open" math that nuclear physics uses. The "Open" features the author hoped to find were either missing or just mathematical artifacts.

In short: The paper is a cautionary note. It tells us that while we can borrow some math from nuclear physics, the specific "leaky" tools they use to track escaping particles don't naturally fit into the current architecture of AI. The "useful" uncertainty in AI is still found in the "Closed" statistical side, not the "Open" dynamic side.

Technical Summary

Technical Summary: Integrating Out, Twice: The Open-System Case That Neural-Network Ensemble Theory Is Missing

Problem Statement
The paper addresses a structural gap in neural-network ensemble theory. While averaging a network over random parameters (marginalizing a Gaussian sector) is mathematically equivalent to the Schur complement of an eliminated block, current ensemble theory only realizes the "closed" case. In this closed scenario, the induced effective object is a real, symmetric, lossless precision matrix (covariance inverse). The paper argues that mainstream learning theory lacks the "open" case: a formalism where eliminating a sector allows probability flux to escape irreversibly, resulting in a non-Hermitian effective generator that conserves and itemizes the lost flux. This open-system structure is fully developed in nuclear reaction theory (specifically the optical model and Feshbach projection) but is absent in machine learning.

Methodology
The author employs a unified algebraic framework based on three elementary tools: the moments of a probability distribution, the algebra of Gaussians, and block matrix inversion. The paper avoids field-theoretic machinery (partition functions, Feynman diagrams) to ensure every step is verifiable by readers familiar with standard linear algebra and statistics.

The methodology proceeds in three stages:

1. Algebraic Unification: The paper demonstrates that marginalizing a neural network ensemble and projecting a scattering problem (Feshbach projection) are instances of the same operation: eliminating a block of a coupled linear system.
2. The "Closed" Dictionary: It establishes a precise correspondence between the closed-case statistics of neural networks and nuclear physics. This includes mapping the Neural Tangent Kernel (NTK) to the Fisher sensitivity kernel, the infinite-width limit to Gaussian-process emulators, and non-Gaussianity to emulator breakdown.
3. The "Open" Export and Testing: The paper attempts to port the open-system machinery (non-Hermitian effective Hamiltonians, flux ledgers, dynamic polarization potentials) to machine learning. It proposes three native ways to induce openness in a network ensemble:
   • Introducing a complex source in the moment generating function.
   • Deforming the parameter measure via a subnormalized survival probability (e.g., rejection of out-of-distribution tokens).
   • Applying Schur complement elimination to network degrees of freedom (retained vs. eliminated blocks).
4. Empirical Validation: The author performs "cheap" numerical tests on three specific learning objects to see if they naturally exhibit the open-system signature:
   • Truncated attention maps (in transformers).
   • Token-level transfer operators (restricted alphabets).
   • Sparse expert routers (mixture-of-experts).

Key Contributions

• The "Open" vs. "Closed" Distinction: The paper clarifies that non-Hermiticity is not merely a feature of complex numbers but the specific signature of an open eliminated sector (one with a continuous spectrum allowing flux escape). Closed eliminations (finite or dissipative) yield real, symmetric Schur complements.
• The Flux Ledger: It defines the "flux ledger" as a conserved, itemized accounting of probability lost to eliminated sectors, derived from the generalized optical theorem. This allows for the decomposition of absorption into direct, breakup, and interference terms.
• The Dictionary of Correspondence: It provides a full dictionary translating nuclear reaction concepts into learning theory, identifying the Neural Tangent Kernel as the Fisher sensitivity kernel and the validity boundary of reduced-basis emulators as the lazy-to-feature transition.
• Native Openness Construction: It demonstrates how to construct an open ensemble natively within learning theory using complex sources or subnormalized measures, without importing nuclear formulas, though it notes this requires non-Gaussianity to manifest operator-level loss.

Results
The paper reports a mostly negative result regarding the applicability of the open-system export to current mainstream learning architectures:

• Truncated Attention: The elimination of tokens in a truncated context window is a closed process. The induced correction is real and symmetric; no non-Hermitian generator arises naturally. Openness must be inserted by hand.
• Token-Level Transfer Operators: While a conserved absorption ledger can be constructed for a restricted alphabet, the "mediated term" (the analog of channel-coupled absorption) is unstable, varying with the partition choice, and appears to be an artifact of the partition rather than a robust property of the model.
• Sparse Routers: In mixture-of-experts models, the "openness" (discarded experts) is driven by the training objective (load balancing) to a high, uninformative floor. The resulting signal is structural (routing sharpness) rather than a meaningful measure of lost flux.
• Epistemic vs. Aleatoric Uncertainty: The paper finds that the "operationally useful" uncertainty (distinguishing hallucination from error) resides in the closed half of the correspondence (ensemble variance/Fisher information), not the open half. The open-system formalism does not currently capture epistemic uncertainty in standard models.
• Structural Barrier: The paper concludes that the open case requires an eliminated sector with a continuous spectrum and wave-like dynamics. Mainstream learning objects are either finite (discrete spectrum) or relaxational (spectrum on the negative real axis), preventing the evaluation point from sitting inside the continuum required to generate a physical anti-Hermitian part.

Significance and Claims
The paper explicitly frames itself as a "note, not a result," and its primary contribution is a map rather than a new theory or a successful application.

• Negative Finding as Value: The main finding is that the open-system export fails for standard learning objects. The paper argues this negative result is valuable because it precisely locates why it fails: the lack of a continuous spectrum and wave-like dynamics in finite or relaxational learning systems.
• Honesty in Analogy: It warns against loose analogies (e.g., "energy landscapes") and insists on precise algebraic identities. It clarifies that while the algebra of elimination is shared, the nature of the eliminated blocks (closed vs. open) dictates the physical properties of the result.
• Future Direction: The paper suggests that for the open-system machinery to be useful in AI, one would need to find or construct architectures that are "wave-like" rather than relaxational, with eliminated sectors unbounded enough to carry a continuum. Until then, the "open case" remains a theoretical possibility rather than a practical tool for current neural networks.

In summary, the paper argues that while neural network ensemble theory has mastered the "closed" integration of parameters, it lacks the "open" integration machinery found in nuclear physics. Attempts to port this machinery to current AI models fail because those models do not possess the necessary spectral properties (continuous spectrum) to support the non-Hermitian, flux-conserving dynamics of the optical model.