Is the existence of unbounded operators a problem for quantum mechanics? In response to Carcassi, Calderon, and Aidala

The paper refutes the claim that Hilbert spaces are unphysical due to unbounded operators, arguing instead that infinite expectation values are not problematic and that replacing Hilbert spaces with Schwartz spaces would introduce greater theoretical issues by excluding meaningful Hamiltonian evolutions.

The Gist

Here is an explanation of the paper using simple language, analogies, and metaphors.

The Big Question: Is the "Rulebook" of Quantum Mechanics Broken?

Imagine you are playing a complex board game called Quantum Mechanics. The game is played on a giant, infinite grid called a Hilbert Space. This grid contains every possible state a particle (like an electron) could be in.

Recently, a group of philosophers (Carcassi, Calderón, and Aidala) looked at this grid and said, "Wait a minute. This grid is too big! It includes some 'states' that make no sense. For example, it allows for a particle to have an infinite amount of energy or position. Since we can't have infinite energy in the real world, this grid must be 'unphysical' (fake). We should throw out the infinite grid and replace it with a smaller, stricter grid called the Schwartz Space that only allows for finite, sensible numbers."

Zhonghao Lu, the author of this paper, says: "Hold on. You don't need to throw out the whole game just because the rulebook allows for some weird, impossible scenarios. In fact, if you shrink the board, you break the game."

Here is Lu's argument broken down into three main points:

---

1. The "Infinite Energy" Scarecrow

The Problem: The critics are worried about "unbounded operators." In plain English, this means there are mathematical tools in quantum mechanics that can calculate values like "position" or "energy." Sometimes, if you use these tools on certain weird states, the answer is Infinity. The critics say, "You can't have a particle with infinite energy, so those states must be fake."

Lu's Analogy: Imagine you are running a weather station. You have a thermometer that can measure temperature.

• The Critics say: "If your thermometer reads 'Infinity' for a specific theoretical scenario, that thermometer is broken, and that scenario is impossible. We should ban the thermometer."
• Lu says: "No. The thermometer isn't broken. It's just that the 'Infinity' reading is a mathematical limit. It's like asking, 'What happens if you keep driving north forever?' The answer is 'You never stop.' That doesn't mean the road is fake; it just means that specific journey is impossible to finish.
  • In quantum mechanics, even if a state has an infinite average energy, you can still calculate the probability of measuring a specific, finite energy. The math still works perfectly fine. The 'infinite' states are just mathematical boundaries, not physical errors."

2. The "Shrinking the Board" Trap

The Proposal: The critics suggest replacing the giant Hilbert Space with the Schwartz Space. Think of this as cutting the board game down to a tiny, safe corner where only "nice" numbers are allowed.

Lu's Counter-Argument: This sounds safe, but it actually breaks the game.

• The Analogy: Imagine you are playing a game of billiards. The critics say, "Let's only play on a table where the balls never go near the edges, because hitting the edge is 'unphysical'."
• The Problem: In the real world (and in quantum mechanics), balls do hit the edges, and they bounce off. If you shrink the table so balls can never hit the edge, you can no longer play the game of billiards as it actually happens.
• The Physics: Lu points out that many real-world forces (like the Coulomb potential that holds atoms together) require the "big board" (Hilbert Space) to work correctly. If you force the system into the "small board" (Schwartz Space), the math says the atoms would instantly fall apart or behave in ways that don't match reality. You would have to ban many real physical laws just to keep the numbers finite.

3. The "What is Real?" Fog

The Philosophical Twist: Lu argues that the word "Physical" is actually a bit vague.

• The Analogy: Think of "Physical" like the word "Healthy."
  • Strict Definition: "Healthy" means you have zero bacteria and zero fatigue. (This is like the critics' view: only finite states are real).
  • Realistic Definition: "Healthy" means you are functioning well enough to live your life, even if you have a cold or get tired. (This is Lu's view: the Hilbert Space is the "realistic" definition).
• Lu suggests that trying to draw a hard line between "Physical" and "Unphysical" is like trying to draw a line between "Day" and "Night." There is a twilight zone.
  • If we demand that every possible state must have finite energy, we might accidentally ban the very laws of physics that make our universe work.
  • He compares this to Quantum Gravity (a theory trying to mix gravity and quantum mechanics). In that field, scientists do need to ban certain states (called the "Hadamard condition") to stop the math from exploding. But in standard quantum mechanics, we don't need to do that yet.

The Conclusion

Lu concludes that we should keep the giant, infinite Hilbert Space.

1. It works: It allows us to predict how particles move and interact perfectly.
2. It's flexible: It handles the "weird" infinite cases without breaking the math.
3. It's necessary: If we shrink the space to only "finite" states, we lose the ability to describe real forces like the electric force in an atom.

The Takeaway: Just because a mathematical map includes a "Here be Dragons" area (infinite values) doesn't mean the map is wrong. It just means that specific part of the map is a theoretical limit, not a place you can actually visit. We don't need to burn the map; we just need to understand how to read it.

Technical Summary

Here is a detailed technical summary of the paper "Is the existence of unbounded operators a problem for quantum mechanics?" by Zhonghao Lu.

1. Problem Statement

The paper addresses a recent claim by Carcassi, Calderón, and Aidala (CCA) that the standard Hilbert space formalism of quantum mechanics is "unphysical." Their argument rests on the existence of unbounded operators (such as position $X$ and momentum $P$).

• The Core Issue: In a Hilbert space $L^2(\mathbb{R})$, completeness ensures that Cauchy sequences of states $\{\psi_i\}$ with finite expectation values $\langle X \rangle_{\psi_i}$ converge to a limit state $\psi$. However, if $\langle X \rangle_{\psi_i} \to \infty$, the limit state $\psi$ lies outside the domain of the operator $X$ ($\psi \notin \text{Dom}(X)$).
• CCA's Conclusion: CCA argues that states with infinite expectation values for physical observables (like position) are "unphysical" because they "do not make physical sense." Consequently, they propose replacing the Hilbert space with the Schwartz space $\mathcal{S}(\mathbb{R})$, which contains only states with finite expectation values for all polynomial operators of position and momentum.
• Historical Context: The paper also revisits earlier concerns by Adrian Heathcote regarding the incompleteness of quantum mechanics: if a state is outside the domain of the Hamiltonian $H$, the Schrödinger equation $i\hbar \frac{d\psi}{dt} = H\psi$ cannot be applied.

2. Methodology

Lu employs a combination of mathematical physics analysis and philosophical conceptual analysis:

• Mathematical Analysis: The author rigorously examines the properties of unbounded self-adjoint operators, spectral theory, Stone's theorem, and the behavior of unitary evolution groups ($e^{-iHt}$) on different function spaces (Hilbert space $L^2$, Schwartz space $\mathcal{S}$, and compact support spaces).
• Counter-Example Construction: The paper constructs specific scenarios (e.g., free evolution of compactly supported wave functions, Coulomb potentials) to test the viability of restricting the state space to $\mathcal{S}(\mathbb{R})$.
• Conceptual Hierarchy Analysis: Lu introduces a hierarchy of "physicality" to analyze the vagueness of the term "physical" in fundamental physics, distinguishing between strict mathematical models and "realistic" physical constraints.
• Comparative Theory: The argument is extended to Quantum Field Theory (QFT) and semiclassical quantum gravity to draw parallels with the Hadamard condition.

3. Key Contributions and Arguments

A. Refutation of the "Unphysical" Status of Infinite Expectation Values

Lu argues that states with infinite expectation values are not unphysical for two main reasons:

1. Operational Definition of Expectation Values: In the standard formalism, expectation values are statistical averages of ensembles, not direct single-shot observations. An infinite expectation value simply implies that the average of long-term measurements does not converge; this is a statistical feature, not a physical impossibility.
2. Probability Distributions Remain Well-Defined: Even if $\psi \notin \text{Dom}(X)$, the spectral decomposition of the operator $X = \int \lambda dP_\lambda$ allows for the calculation of probability distributions for measurement outcomes. The probability of a result falling in a Borel set $\epsilon$ is $\int_\epsilon dP_\lambda$, which remains well-defined regardless of whether the first moment (expectation value) is finite.

B. Resolution of the Schrödinger Equation "Incompleteness"

Regarding the concern that the Schrödinger equation fails for states outside $\text{Dom}(H)$:

• Lu invokes Stone's Theorem, which states that a self-adjoint operator $H$ generates a unique one-parameter unitary group $U(t) = e^{-iHt}$ defined on the entire Hilbert space.
• The time evolution $\psi(t) = U(t)\psi(0)$ is well-defined and deterministic for all initial states, even those where $H\psi$ is undefined. The differential form (Schrödinger equation) is merely a special case valid only when $\psi \in \text{Dom}(H)$. Therefore, restricting the state space to ensure differentiability is unnecessary and undesirable.

C. Critique of the Schwartz Space ($\mathcal{S}(\mathbb{R})$) Proposal

Lu demonstrates that replacing $L^2(\mathbb{R})$ with $\mathcal{S}(\mathbb{R})$ introduces severe physical and mathematical problems:

1. Arbitrariness of Constraints: There is no physical justification for requiring finite expectation values for polynomials ($x^n p^m$) while allowing infinite values for exponentials ($e^x$). Both are derived from the same measurement apparatus data.
2. Failure of Dynamical Closure: The Schwartz space is not closed under unitary evolution for many physically meaningful Hamiltonians.
   • Example 1 (Free Particle): A wave function with compact support (a subset of $\mathcal{S}$) evolves under a free Hamiltonian $H_0$ into a state with infinite support, leaving the space of compactly supported functions.
   • Example 2 (Coulomb Potential): For potentials $V(x)$ with negative powers (e.g., $1/x$), the time derivative of momentum expectation values$\frac{d}{dt}\langle p \rangle$may diverge for states in$\mathcal{S}(\mathbb{R})$, causing the evolved state to exit the Schwartz space.
3. Loss of Essential Self-Adjointness: Restricting operators to $\mathcal{S}(\mathbb{R})$ complicates the definition of observables. While $\mathcal{S}(\mathbb{R})$ is dense, determining whether a symmetric operator on $\mathcal{S}(\mathbb{R})$ is essentially self-adjoint (and thus has a unique self-adjoint extension) often requires reference to the larger Hilbert space structure, undermining the goal of an independent formulation.

D. Analysis of "Physicality" and Modality

Lu proposes a hierarchy of "physical possible worlds" to contextualize the debate:

• Hierarchy I (Strong Determinism): Only one world is possible.
• Hierarchy II (Mathematical Models): All models satisfying the fundamental equations are physical.
• Hierarchy III (Realistic Models): Models satisfying equations plus "realistic" constraints (e.g., finite energy, finite expectation values).
• Conclusion: Lu argues that "physicality" in Hierarchy III is vague and arbitrary. What CCA treats as a "realistic" constraint (finite expectation values) is actually a mathematical restriction that excludes valid physical dynamics. In more advanced theories (like QFT), constraints like the Hadamard condition are necessary for mathematical consistency (renormalization), not just "realism," suggesting that the boundary between physical and unphysical is theory-dependent and often vague.

4. Results

• Unbounded operators do not render quantum mechanics unphysical. The existence of states with infinite expectation values is a mathematical feature that does not hinder the calculation of probabilities or the deterministic evolution of the system via unitary groups.
• The Schwartz space is an inadequate replacement for Hilbert space. It fails to be invariant under the time evolution of standard physical Hamiltonians (including the Coulomb potential), thereby excluding physically meaningful dynamics.
• The distinction between "physical" and "unphysical" states is not absolute. It depends on the mathematical structure of the theory and the specific constraints required for consistency (as seen in the Hadamard condition in QFT), rather than a simple requirement of finite expectation values.

5. Significance

• Defense of Standard Formalism: The paper provides a robust defense of the standard Hilbert space formulation of quantum mechanics against recent philosophical critiques, clarifying that the domain issues of unbounded operators are resolved by spectral theory and unitary evolution.
• Clarification of Mathematical Rigor: It highlights the dangers of imposing ad-hoc restrictions on state spaces that break the closure properties required for dynamical evolution, emphasizing that "physical" theories must accommodate the mathematical necessities of their own dynamics.
• Bridging QM and QFT: By connecting the debate to the Hadamard condition in Quantum Field Theory, the paper suggests that the "physicality" of states is a nuanced concept that evolves with the complexity of the theory, moving from simple "realism" to requirements of mathematical consistency in interacting field theories.
• Philosophical Insight: It challenges the notion of "physicality" as a clear-cut binary, proposing instead a hierarchical and context-dependent view of what constitutes a possible physical world.