⚛️ quantum physics

Quantum description of reality is epistemically incomplete

This paper establishes that the operational quantum description of reality is epistemically incomplete by proving that any classical hidden-variable theory preserving empirical preparation properties must satisfy a specific distinguishability equality, which quantum mechanics violates, thereby certifying the existence of inaccessible ontic structure that underpins quantum communication advantages.

Original authors: Anubhav Chaturvedi, Marcin Pawłowski, Debashis Saha

Published 2026-03-24

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Anubhav Chaturvedi, Marcin Pawłowski, Debashis Saha

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to describe a mysterious object to a friend, but you can only describe it by how it behaves when you poke, push, or shine a light on it. This is how quantum physics works: we describe reality based on what we can measure (the "operational" view).

But for nearly a century, physicists have asked a deep question: Is this description complete? Or, is there a hidden "real" layer underneath (like a secret engine inside a car) that explains why the car behaves the way it does, even if we can't see the engine?

This paper argues that the quantum description is incomplete. It proves that no matter how you try to build a "hidden engine" (a classical model) to explain quantum behavior, you will always have to hide some of the engine's power to make it match our observations.

Here is the breakdown using simple analogies.

1. The Game: "Guess the Box"

Imagine Alice has a set of $N$ different boxes. She picks one secretly and sends it to Bob. Bob's job is to guess which box it is.

The Quantum Reality: Quantum mechanics tells us exactly how good Bob can be at this game. Sometimes he can guess perfectly; sometimes he has to guess with a certain probability of being right.
The "Hidden Engine" Theory: A classical realist would say, "There must be a hidden label inside the box that tells Bob exactly what it is. If Bob could see that label, he would be even better at guessing."

2. The New Rule: "Epistemic Completeness"

The authors introduce a new rule called Epistemic Completeness.

The Rule: If a "hidden engine" theory is truly complete, then the power Bob has if he can see the hidden label should be exactly the same as the power he has in the real world (where he can't see the label).
The Catch: If the hidden engine gives Bob more power than he actually has in the real world, then the theory is incomplete. It means the theory has "extra horsepower" that it is forced to hide (or "fine-tune") to match reality.

3. The "Magic Balance Scale" (The Main Discovery)

The authors found a specific mathematical balance scale that must be perfectly flat if the world is classical (complete).

They looked at two ways Bob can play the guessing game with $N$ boxes:

Pairwise Guessing: Bob is shown two boxes and asked, "Which one is it?" (He guesses between pairs).
Set Guessing: Bob is asked, "Is it in this group of 3? Or this group of 4?" (He guesses between larger groups).

The Theorem: In any world where reality is fully explained by hidden variables (a complete classical world), the average success rate of Pairwise Guessing must exactly equal the average success rate of Set Guessing.

Think of it like a financial ledger. In a complete system, your "Income" (Pairwise success) must exactly match your "Expenses" (Set success). If they don't match, you are hiding money.

4. The Quantum Violation: "The Ledger is Unbalanced"

When the authors tested this with real quantum particles (like photons or electrons), the scale tilted.

Quantum particles were better at Pairwise Guessing than Set Guessing (or vice versa, depending on the setup).
The Result: The "Ledger" was unbalanced. This proves that Quantum Mechanics is Epistemically Incomplete.
The Meaning: There is a hidden "communication power" inside the quantum world that is stronger than what we can observe. To make the math work, any hidden-variable theory must artificially suppress this extra power. It's like a car that has a V12 engine but is forced to drive at 30 mph; the engine is there, but it's being throttled down to match the speed limit.

5. Why This Matters (The "So What?")

This isn't just abstract math; it has real-world consequences:

Quantum Advantage: Because the quantum "hidden engine" has extra power, quantum computers and communication systems can do things that classical computers simply cannot, even if we put strict limits on them. The violation of the balance scale is a certificate of this advantage.
Coherence and Incompatibility: The fact that the scale is unbalanced proves that quantum states are "coherent" (they are in a superposition, not just a hidden list of options) and that quantum measurements are "incompatible" (you can't measure everything at once without disturbing the system).
Robustness: The authors showed that this "unbalanced ledger" doesn't disappear even if the quantum system is noisy or imperfect. It's a fundamental feature of the universe, not a fluke of a perfect experiment.

6. The "Perfect Detective" (Kochen-Specker Model)

The paper also looked at a famous "hidden variable" theory called the Kochen-Specker model.

Usually, we just say "Hidden variables fail."
But here, the authors calculated exactly how much extra power the hidden variables have.
They found that for specific quantum setups (like the "Trine" and "Tetrahedral" arrangements of particles), the Kochen-Specker model exactly matches the amount of hidden power required to explain the quantum results.
The Analogy: It's like finding a detective who can explain a crime scene perfectly, but only if you admit the detective has a secret superpower that the police force doesn't know about. The paper quantifies exactly how big that superpower is.

Summary

The paper proves that you cannot fully explain quantum mechanics using a simple, hidden "realist" description without hiding some of that description's power.

If you try to build a classical model of the quantum world, you will find that your model has "too much power" compared to what we actually observe. To make it fit, you have to artificially restrict it. This "restriction" is the proof that the quantum description is epistemically incomplete—there is more to reality than what we can operationally measure, and that "more" is the source of quantum magic.

1. Problem Statement

The paper addresses a foundational question in quantum mechanics: Is the operational quantum description of reality complete? Specifically, it asks whether the empirically accessible properties of a finite set of quantum preparations can be exactly reproduced by an underlying ontological (hidden-variable) description, or if any such completion inevitably requires "hidden structure" that is inaccessible at the operational level.

The authors move beyond traditional tests (like Bell nonlocality or contextuality) which often focus on specific operational features. Instead, they propose a broader criterion: Epistemic Completeness. A theory is epistemically complete if every empirically accessible preparation property (defined via optimal one-way communication tasks) is preserved exactly by the corresponding "not-fine-tuned" ontic quantities. If an ontological model requires fine-tuning (restricting valid decoders) to match operational statistics, the theory is deemed epistemically incomplete.

2. Methodology

A. Defining Empirical Preparation Properties

The authors define empirical properties of a preparation set $\mathcal{P} = \{P_1, \dots, P_n\}$ through one-way communication tasks.

Task: Alice prepares a state $P_x$ based on a label $x$ . Bob performs a measurement and outputs a guess $k$ .
Metric: The performance is measured by the optimal success probability (or task payoff) $S^{(O)}_n(\mathcal{P})$ , optimized over all allowed measurements in the operational theory.
Key Family: The paper focuses on Set-Distinguishability Tasks ( $D_{n,m}$ $D_{n, m}$ ), where Bob must output an $m$ $m$ -element subset containing the true label $x$ $x$ .
- $m=1$ : Ordinary distinguishability (exact identification).
- $m=n-1$ : Antidistinguishability (exact exclusion).
- Intermediate $m$ : Interpolates between identification and exclusion.

B. Defining Epistemic Completeness

The authors introduce a comparison between the operational level and the ontic level:

Operational Level: Performance is limited by the specific measurements available in the theory.
Ontic Level: If the hidden state $\lambda$ were accessible, the decoder could use any response scheme compatible only with positivity and normalization (no fine-tuning).
Criterion: A theory is epistemically complete if, for every finite preparation set, the operational task values equal the not-fine-tuned ontic task values.

C. The Equality Witness

The core of the methodology is the derivation of an exact equality that must hold in any epistemically complete theory. The authors define two averaged quantities derived from the set-distinguishability family:

Average Pairwise Distinguishability ( $D_n$ ): The average success probability of distinguishing any two states in the set.
Average Set-Distinguishability ( $S_n$ ): The average success probability across all subset sizes $m \in \{1, \dots, n-1\}$ .

Theorem: In any epistemically complete theory, for any finite preparation set:
$D_n = S_n$
This equality arises from a combinatorial identity regarding ordered lists of real numbers applied pointwise to the ontic probability distributions.

3. Key Contributions

Formalization of Epistemic Completeness: The paper provides a rigorous definition of classicality that requires the exact preservation of the entire family of empirical communication-task quantities, rather than just a single witness (like distinguishability or noncontextuality).
Universal Equality Theorem: The authors prove that $D_n = S_n$ is a necessary condition for epistemic completeness. This serves as a "no-go" theorem: any deviation certifies incompleteness.
Quantitative Witness: The deviation $\Delta = D_n - S_n$ is not just a binary test. Its magnitude provides a lower bound on the amount of "hidden ontic communication power" that any completion must conceal.
Unification of Classicality Notions: The paper demonstrates that epistemic completeness implies Bounded Ontological Distinctness and Preparation Noncontextuality, placing it at the top of a hierarchy of classicality principles.
Sharpness via Kochen-Specker Model: The authors show that the famous Kochen-Specker $\psi$ -epistemic model for qubits exactly saturates the hidden excess implied by the quantum violation, making the witness quantitatively sharp for canonical ensembles.

4. Key Results

A. Quantum Violations

Quantum theory violates the equality $D_n = S_n$ in both directions:

Positive Deviation ( $D_n > S_n$ ):
- Trine Ensemble ( $n=3$ ): Three pure qubit states symmetrically spaced on the Bloch sphere.
  - Result: $\Delta \approx 0.0997$ .
  - This is the exact maximizer for $n=3$ .
- Tetrahedral Ensemble ( $n=4$ ): Four pure qubit states pointing to the vertices of a regular tetrahedron.
  - Result: $\Delta \approx 0.1454$ .
  - This is the numerically certified maximizer for $n=4$ .
Negative Deviation ( $S_n > D_n$ ):
- Achieved by mixed-state qutrit ensembles (for $n=3$ ) and higher-dimensional ensembles.
- Result: $\Delta \approx -0.0277$ for $n=3$ .

B. Robustness

The violations are robust against noise and information leakage:

Visibility Families: The positive deviation persists for arbitrarily low positive visibility (depolarization) of the Trine and Tetrahedral ensembles.
Leakage Families: The deviation persists even when an orthogonal branch is added that partially reveals the preparation label, up to the point of complete disclosure.

C. Operational Implications

Any nonzero deviation implies:

Quantum Communication Advantage: Under a benchmark constraint (fixing one quantity, e.g., $S_n$ ), the quantum preparation outperforms any unrestricted classical communication model in the other task (e.g., $D_n$ ).
Coherence Witness: Since finite-dimensional commuting quantum theories are epistemically complete, any deviation certifies that the preparation set possesses genuine quantum coherence (non-commutativity).
Measurement Incompatibility: Any realization of the task values with a nonzero deviation requires an incompatible (non-jointly measurable) set of measurements.

D. Numerical Characterization

Using Semidefinite Programming (SDP) hierarchies and see-saw methods, the authors:

Certified the Trine and Tetrahedral ensembles as the optimal positive-direction maximizers for $n=3$ and $n=4$ .
Found evidence that the maximal positive deviation increases with the number of preparations ( $n$ ), suggesting an open question regarding the absolute maximal epistemic incompleteness of quantum theory.

5. Significance

Foundational Insight: The paper establishes that the quantum description is fundamentally incomplete not just in specific scenarios, but structurally: the operational statistics of preparation sets cannot be fully explained by any hidden-variable model without "hiding" communication power.
Resource Theory: It identifies the deviation $\Delta$ as a resource. A nonzero $\Delta$ guarantees a communication advantage over classical resources under natural constraints, linking foundational incompleteness directly to practical quantum advantage.
Quantitative Precision: Unlike many "no-go" theorems that are qualitative, this work provides an exact accounting identity. The Kochen-Specker model's ability to saturate the bound demonstrates that the "missing" information is precisely quantifiable.
Unifying Framework: It subsumes previous notions of classicality (like preparation noncontextuality and bounded ontological distinctness) under a single, stronger umbrella, clarifying the logical hierarchy of classical explanations.

In summary, the paper proves that quantum mechanics is epistemically incomplete by showing that the average pairwise distinguishability of quantum states does not equal their average set-distinguishability. This inequality serves as a robust, quantitative witness of quantum coherence, measurement incompatibility, and communication advantage, revealing that any classical completion of quantum theory must conceal a specific, measurable amount of hidden ontic power.