Physics-Aware, Shannon-Optimal Compression via Arithmetic Coding for Distributional Fidelity

This paper proposes a physics-aware, information-theoretic framework that uses arithmetic coding to establish an absolute, bit-quantified metric for assessing the distributional fidelity of datasets by measuring the irreducible excess codelength relative to a physics-informed probabilistic model.

The Gist

Imagine you are a detective trying to figure out if a painting is a genuine masterpiece by a famous artist or a clever forgery.

Usually, detectives look for specific clues: the type of brushstroke, the chemical composition of the paint, or the signature. But what if the forgery is so good that it passes all those specific tests? You need a different kind of test—one that looks at the "soul" of the painting as a whole.

This paper proposes a new kind of detective work for data, specifically for complex scientific data like images from particle detectors or AI-generated simulations. Instead of looking for specific flaws, the authors use compression (like zipping a file) as a truth meter.

Here is the breakdown of their idea using simple analogies:

1. The Problem: How do we know if data is "Real"?

Scientists often generate massive amounts of data, either from real experiments (like smashing particles together) or from AI simulations. They need to know: Does this AI simulation actually look like the real thing?

Current methods are like checking a painting by counting the number of blue pixels. They compare specific features. But if the AI is smart, it can fake those specific features while missing the deeper, invisible connections between them. These methods often rely on arbitrary choices (like "let's count blue pixels") that might miss the big picture.

2. The Solution: The "Zip File" Test

The authors suggest a different approach: Try to compress the data.

Think of data compression like packing a suitcase for a trip.

• Real Data (The Genuine Painting): If you have a suitcase full of clothes that all belong to the same person, they fit together perfectly. You can fold them tightly, tuck socks into shoes, and roll shirts. The suitcase is small and efficient. This is because the clothes have a natural "logic" or pattern to them.
• Fake Data (The Forgery): If you try to pack a suitcase with random items that don't belong together (a winter coat, a swimsuit, and a pair of boots all mixed up), they won't fit well. You'll have empty spaces, and the suitcase will be huge.

In the world of computers, Arithmetic Coding is the ultimate packer. It is a mathematical tool that knows exactly how to fold data based on the rules of probability.

3. The "Physics-Aware" Packer

The genius of this paper is that they didn't just use a generic packer (like the standard zip or gzip on your computer). They built a Physics-Aware Packer.

Imagine a packer who knows the rules of the universe.

• If you give them a real particle detector reading, they know: "Ah, when a particle hits this specific layer, it usually hits this neighbor layer too." They pack those two things together tightly because they are related.
• If you give them a fake simulation that got the physics wrong (e.g., the particle hit the neighbor layer when it shouldn't have), the packer gets confused. "Wait, that doesn't make sense!" they say. Because the data breaks the rules of physics, the packer can't fold it efficiently. The "suitcase" (the file size) ends up being larger than it should be.

4. The Result: Measuring "Badness" in Bits

The paper introduces a new way to measure how "fake" the data is.

• The Metric: They measure the extra space the fake data takes up compared to the real data.
• The Unit: They measure this in bits (the smallest unit of computer information).
• The Meaning: If the fake data takes up 0 extra bits, it is indistinguishable from the real thing under the laws of physics. If it takes up 10 extra bits, it means the data is "clunky" and violates the natural patterns of the universe.

This is a "Goldilocks" metric. It's not just "pass/fail." It tells you exactly how much the data is wrong, in a language (bits) that is absolute and universal.

5. Why This Matters

• No Arbitrary Choices: Unlike other methods where you have to decide "what features to look at," this method looks at everything at once. If the data is weird, the suitcase gets big.
• Global View: It doesn't just check one part of the data; it checks the whole story.
• Better than Standard Tools: The authors showed that their "Physics-Aware Packer" could spot tiny errors in the data that standard compression tools (like gzip) and other statistical tests (like MMD) completely missed. It's like having a detective with X-ray vision who can see the invisible connections between particles.

The Bottom Line

This paper turns data compression from a tool for saving space into a scientific instrument for measuring truth.

If you can compress your data tightly using the rules of physics, your data is good. If the file size balloons because the data doesn't follow the rules, you know something is wrong. It's a simple, elegant, and powerful way to tell if a simulation is a masterpiece or a forgery.

Technical Summary

1. Problem Statement

Modern scientific analysis, particularly in high-energy physics and generative AI, faces a critical challenge: assessing the distributional fidelity of synthetic or simulated datasets against real observations.

• Limitations of Existing Methods: Current approaches (e.g., Kullback-Leibler divergence, Maximum Mean Discrepancy, kernel-based distances) are often relative or heuristic. They require external choices of test statistics, kernels, or feature spaces that may not capture intrinsic physical correlations.
• The Gap: There is a lack of an absolute, physically grounded standard for fidelity. Existing metrics often lack an intrinsic scale, making it difficult to define what constitutes a "perfect" match versus a "good enough" one. Furthermore, many methods rely on low-dimensional summaries or explicit density models that struggle with high-dimensional, multimodal data.

2. Methodology

The authors propose an information-theoretic framework where lossless compression serves as an operational measure of dataset fidelity.

Core Concept: Arithmetic Coding (AC)

Instead of using compression merely for data reduction, the paper utilizes Arithmetic Coding as a measurement instrument.

• Shannon Optimality: AC maps a sequence of symbols to a binary string with a length $\ell(x) \approx -\log_2 q(x)$, where $q(x)$ is a reference probability model.
• The Metric: The excess codelength ($\Delta L$) between a test dataset and a reference dataset, when encoded by the same fixed model, quantifies the distributional mismatch.
  • If a dataset $C$ is generated from the same distribution as the reference $B$, $\Delta L \approx 0$.
  • If $C$ contains miscalibrations, biases, or modeling errors, it incurs an irreducible excess codelength relative to the Shannon-optimal limit defined by the physics.
  • Mathematically: $\Delta L \approx D_{KL}(p_{test} || q_{ref})$, where $p$ is the true distribution and $q$ is the reference model.

Physics-Aware Probabilistic Representation

The framework employs a physics-informed factorization of the data to construct the probability model $q(x)$:

1. Data Source: Simulated electromagnetic calorimeter data (CLAS12 detector) including hits (ADC values, strip IDs) and particle kinematics ($p_x, p_y, p_z$).
2. Factorization: The model separates occupancy (hit presence) from attributes (strip ID, ADC amplitude) and conditions hit distributions on particle momentum ($|p|$).
   • Unconditional Model: $q(hits) \approx \prod q(occ)q(strip|occ)q(adc|occ)$.
   • Conditional Model: $q(hits| |p|) \approx \prod q(occ| |p|)q(strip|occ, |p|)q(adc|occ, |p|)$.
3. Lossless Guarantee: The encoding is strictly lossless and invertible, ensuring that no information is lost during the compression-decompression cycle. This validates that the codelength reflects statistical structure, not data degradation.

Experimental Design

• Data Splits: The dataset is split into three independent sets to prevent data leakage:
  • Training ($A$): Used to learn the CDF tables (probability model).
  • Baseline ($B$): An independent sample from the same distribution as $A$ (used as the "truth" reference).
  • Perturbed ($C_\epsilon$): A sample derived from $B$ with controlled distortions (e.g., ADC scale perturbations).
• Comparison: The study compares the proposed AC method against the Maximum Mean Discrepancy (MMD), a standard kernel-based two-sample test.

3. Key Contributions

1. Absolute Fidelity Metric: Introduces a metric expressed in bits per event that provides a physically meaningful, absolute scale for consistency. Zero excess codelength corresponds to perfect consistency with the underlying physical model.
2. Global and Additive: Unlike methods relying on specific observables, AC probes the full joint distribution. The codelength is additive, allowing researchers to decompose fidelity errors by detector subsystem (e.g., PCAL vs. ECIN) or data component (hits vs. kinematics).
3. Shannon-Optimal Baseline: Demonstrates that physics-aware AC achieves compression ratios close to the theoretical Shannon limit, outperforming general-purpose compressors (like gzip) by up to 1.6x–2x.
4. Model-Conditional Interpretation: Clarifies that fidelity is defined relative to a specific physical representation. Enriching the model (e.g., adding kinematic conditioning) changes the "definition" of fidelity, allowing for sharper sensitivity to specific physical correlations.

4. Key Results

• Invertibility: The compression-decompression cycle preserves all detector-level observables (ADC distributions, hit multiplicities) exactly, confirming the method is lossless.
• Compression Efficiency:
  • AC reduced data size by ~13.5x (unconditional) and ~13.2x (conditional) compared to raw data.
  • AC outperformed gzip-9 by a factor of 1.6x, proving that generic compressors fail to exploit the structured, physics-driven regularities in detector data.
• Sensitivity to Perturbations:
  • Conditional AC detected statistically significant deviations (p < 0.05) at ADC scale perturbations as small as $\epsilon \approx 10^{-4}$.
  • Unconditional AC required larger perturbations ($\epsilon \gtrsim 10^{-2}$).
  • MMD remained insensitive to small perturbations and only showed significant deviation at $\epsilon \gtrsim 4 \times 10^{-3}$.
• Bit-Budget Decomposition: The study successfully decomposed the codelength into contributions from occupancy, strip indices, and ADC amplitudes, showing that ADC amplitudes dominate the information content. Conditioning on kinematics reduced the entropy of occupancy but increased the complexity of ADC modeling, providing a nuanced view of where physical correlations reside.

5. Significance and Implications

• New Paradigm for Validation: This work elevates lossless compression from a data-reduction tool to a quantitative scientific instrument. It allows physicists to validate simulations and generative models against real data without hand-crafted observables.
• Interpretability: The "bit penalty" provides a transparent mapping between physical errors (e.g., calibration drift) and their information-theoretic cost. This offers a level of interpretability often missing in "black-box" statistical tests.
• Complementarity: The method is complementary to kernel-based tests. While MMD tests for equality in an abstract feature space, AC tests for typicality under a fixed physical model. A dataset can be indistinguishable by MMD but fail the compression test if it violates specific physical correlations encoded in the model.
• Future Applications: The framework supports anomaly detection, calibration studies, and the validation of fast simulation pipelines. It suggests a future where information-theoretic metrics are foundational for the analysis of high-dimensional experimental data.

In summary, the paper establishes that arithmetic coding, when guided by physical knowledge, provides a rigorous, absolute, and interpretable standard for assessing the fidelity of complex scientific datasets.