Physics-driven Comparative Analysis of Various Statistical Distance Metrics and Normalizing Functions

This paper presents a data-driven comparative analysis of various statistical distance metrics and normalizing functions using electron and photon events from a decaying Kr-83 isotope to evaluate the stability of a dimensionless Parameter of Interest under different conditions.

The Gist

Imagine you are a detective trying to tell the difference between two types of suspects: Electrons (charged particles) and Photons (light particles). In the world of physics, these two leave behind very specific "footprints" when they hit a detector. But sometimes, the footprints look a little blurry, or the camera isn't perfect.

This paper is essentially a tool test. The author, N. Fuad, is asking: "When I have two piles of footprints (data), which mathematical ruler is the best at measuring how different they really are?"

Here is the breakdown of the story, using simple analogies:

1. The Crime Scene: The Detector

The "detective" used a giant, super-cold camera made of Germanium (a semiconductor) to watch an isotope called Krypton-83 decay.

• The Setup: Think of this camera as a high-speed microphone. When a particle hits it, it creates a sound wave (a signal).
• The Difference:
  • Electrons are like a sprinter who stops abruptly. They hit the camera and stop almost instantly, creating a signal that rises very sharply.
  • Photons are like a marathon runner who slows down gradually. They travel further inside the camera before stopping, creating a signal that rises slowly.
• The Goal: The team needed to separate these two groups perfectly. To do that, they created a "score" (a number between 0 and 1) for every event based on how sharp the signal was. This score is their Parameter of Interest (PoI).

2. The Problem: Too Many Rulers

In math and science, there are dozens of ways to measure "distance" or "difference" between two groups of data. Some are called Hellinger, some Wasserstein, some Kolmogorov-Smirnov, and so on.

Imagine you have two piles of sand.

• Ruler A measures the difference in the height of the piles.
• Ruler B measures how much sand you have to move to turn one pile into the other.
• Ruler C measures the difference in the shape of the piles.

The problem is, some of these rulers are finicky.

• If you change the size of the grains of sand (discretization), some rulers give wildly different answers.
• If you only have a few grains of sand (low statistics), some rulers break or give nonsense numbers.
• Some rulers are so sensitive that they say two piles are "completely different" even if they are just slightly different.

3. The Experiment: The "Normalization" Trick

The author realized that some of these rulers produce numbers that are too big or too small to compare fairly. So, they introduced Normalizing Functions.

Think of this like a translator or a compressor.

• Imagine a ruler that measures distance in "light-years" (huge numbers) and another in "inches" (tiny numbers). It's hard to compare them.
• The author invented special "squeeze functions" (like $n(x)$) that take any huge number and squash it down into a neat, tidy box between 0 and 1.
  • 0 means "Identical."
  • 1 means "Completely Different."

They tested four different "squeezers" to see which one made the rulers play nice together.

4. The Results: Who Won?

After running thousands of tests with their electron and photon data, here is what they found:

• The Unreliable Ones:

  • Wasserstein-2 ($W_2$): This ruler is very sensitive to how you slice the data. If you change the grain size of your sand, this ruler panics and gives a different answer.
  • Fisher-Rao & $L_\infty$: These are great if you have perfect data, but if you have a small sample size (few events), they become unstable and unreliable.
  • The "Saturators": Some rulers (like $W_2$ and $L_\infty$) hit the "1.0" ceiling too easily. They say "These are totally different!" even when they are just mostly different. They lose the ability to tell the difference between "very different" and "maximally different."

• The Winner: The $\sqrt{JS}$ Distance

  • The Square Root of Jensen-Shannon ($\sqrt{JS}$) distance was the champion.
  • Why? It was the most stable. Whether they changed the size of the data grains, the number of events, or which "squeezer" they used, this ruler gave consistent, reliable answers.
  • It didn't get confused by small changes in the data, and it didn't break when the sample size was small.

5. The Takeaway

The paper concludes that if you are trying to compare two probability distributions (like electron vs. photon signals) in a noisy, real-world environment:

1. Don't just pick a ruler at random; some are too sensitive to the "noise."
2. Use the $\sqrt{JS}$ distance. It's the most robust tool in the toolbox.
3. If you need to squash big numbers into a 0-to-1 range, you can use simple mathematical "squeezers," but they all seem to work about the same for this specific job.

In short: The author tested a bunch of mathematical rulers to see which one is the best at telling apart electrons from photons. They found that the $\sqrt{JS}$ ruler is the most reliable detective, while the others tend to get confused by the details of the experiment.

Technical Summary

1. Problem Statement

In scientific analysis, machine learning, and hypothesis testing, there is a ubiquitous need to quantify the dissimilarity between two Probability Density Functions (PDFs) or Probability Mass Functions (PMFs). While numerous distance metrics and divergence measures exist (e.g., Hellinger, Wasserstein, Jensen-Shannon), there is no standardized consensus on which metric is most robust for specific physical applications.

Key challenges addressed in this paper include:

• Metric Variability: Different metrics often yield vastly different "distances" for the same pair of distributions.
• Normalization Sensitivity: Many metrics produce unbounded values or values that saturate at 1, making them insensitive to differences between "fully disjoint" and "maximally disjoint" sets.
• Stability Issues: Metrics may behave unpredictably under variations in sample size (low statistics) or discretization length (binning).
• Lack of Physical Context: Many comparisons are purely mathematical; this study seeks to ground the comparison in real-world experimental physics data.

2. Methodology

A. Data Source and Physics Context

The study utilizes experimental data from a High-Purity Germanium (HPGe) spectrometer (PPC type) operating under cryo-vacuum conditions.

• Source: A decaying $^{83}$Kr isotope.
• Event Types: The dataset distinguishes between electron events (charged particles) and photon events (neutral particles).
• Physical Distinction: Electrons deposit energy very close to the surface (stopping within $\sim\mu$m), creating a sharp rise in the signal waveform. Photons penetrate deeper ($\sim 100\mu$m), resulting in a slower rise time.
• Parameter of Interest (PoI): The authors define a dimensionless parameter $x$ based on the sharpness of the waveform's rising edge:
  $$x = \max\left(\frac{ds(t)/dt}{E}\right)$$
  where $s(t)$ is the signal waveform and $E$ is the energy. This value is rescaled to the range $[0, 1]$.
• Distribution Generation: PMFs are generated from the $x$ values of 6,982 electron events and 14,146 photon events. The distributions are observed to be disjoint but not maximally disjoint.

B. Metrics Evaluated

The paper compares seven specific distance metrics between the electron and photon PMFs:

1. Hellinger Distance ($H$)
2. Wasserstein-1 Distance ($W_1$)
3. Wasserstein-2 Distance ($W_2$)
4. Square Root of Jensen-Shannon Divergence ($\sqrt{JS}$)
5. $L_\infty$ Norm (Chebyshev Distance)
6. Kolmogorov-Smirnov Distance ($KS$)
7. Fisher-Rao Distance ($FR$)

C. Normalization Strategy

Since some metrics (like $W_1$) are unbounded while others (like $FR$) are bounded by $\pi/2$ or 1, the authors propose a set of normalizing functions $n(x)$ to map all metric outputs to a standard range $[0, 1)$.

• Proposed Properties for $n(x)$: Bounds ($0 \to 1$), Bijectivity, Monotonicity, and Metric Preservation (if $d$ is a metric, $n \circ d$ must also be a metric).
• Functions Tested:
  1. $n_1(x) = \frac{\log(1+x)}{1+\log(1+x)}$
  2. $n_2(x) = \frac{x}{1+x}$
  3. $n_3(x) = 1 - e^{-x}$
  4. $n_4(x) = \frac{2}{\pi}\arctan(x)$
• Control: A "no normalization" case ($n_0$) was also tested.

D. Stability Analysis

The robustness of each metric was tested against:

• Discretization Length: Varying the bin size of the PMFs.
• Sample Size: Reducing the number of events to simulate low-statistics scenarios.

3. Key Results

A. Metric Performance and Saturation

• Saturation Issues: Metrics like Hellinger, KS, and Fisher-Rao frequently saturated at a value of 1.0, indicating they could not distinguish between distributions that were merely disjoint versus those that were maximally disjoint.
• Unbounded Instability: $W_1$ and $L_\infty$ were the least prone to saturation but were highly unstable under changes in discretization and low sample sizes.
• The Winner ($\sqrt{JS}$): The Square Root of Jensen-Shannon distance emerged as the most reliable metric. It maintained stability across discretization lengths and sample sizes without saturating prematurely.

B. Impact of Normalization

• Standard Deviation: Manually defined normalizing functions ($n_1$–$n_4$) generally reduced the standard deviation of the measurements compared to raw, unnormalized metrics. This suggests normalization brings different metrics into closer agreement.
• Metric Sensitivity:
  • Least Impacted: Hellinger and $\sqrt{JS}$ distances showed minimal change regardless of the normalization function chosen.
  • Most Impacted: $L_\infty$ and Fisher-Rao distances were highly sensitive to the choice of normalization function.
• Specific Behavior: $W_2$ and $\sqrt{JS}$ only saturated under the $n_3(x)$ function, whereas $W_2$ was found to be extremely unstable regarding discretization and statistics.

C. Visual and Statistical Observations

• Figure 9 (Discretization): $W_2$ was the most sensitive to changes in bin size, while Hellinger, $\sqrt{JS}$, and Fisher-Rao remained stable.
• Figure 10 (Sample Size): $\sqrt{JS}$, $L_\infty$, and Fisher-Rao were observed to be the most unstable under low statistics (small $N$), though $\sqrt{JS}$ recovered better than the others in the overall stability analysis.

4. Key Contributions

1. Physics-Driven Benchmark: The paper provides a rare, systematic comparison of statistical distance metrics using real, noisy experimental data (HPGe detector signals) rather than synthetic mathematical distributions.
2. Definition of Normalization Criteria: It formally defines the properties a normalizing function must possess to preserve metric axioms (specifically the triangle inequality) and proposes four specific functions to test these properties.
3. Identification of Optimal Metric: It identifies $\sqrt{JS}$ as the superior metric for this specific class of problems (distinguishing particle types in spectroscopy) due to its balance of non-maximality preservation and stability.
4. Normalization Insights: It demonstrates that while normalization reduces variance between metrics, the choice of function matters significantly for certain metrics (like $L_\infty$) but less for others (like $\sqrt{JS}$).

5. Significance

This work is significant for the fields of nuclear physics, particle detection, and machine learning applied to scientific data.

• For Experimentalists: It provides a guideline for selecting robust statistical tools to distinguish between signal and background events (e.g., electron vs. photon discrimination) in low-statistics or high-noise environments.
• For Data Scientists: It highlights the pitfalls of using unbounded metrics without normalization and offers a framework for testing metric stability against discretization and sampling errors.
• Generalizability: The proposed "Parameter of Interest" (waveform sharpness) and the comparative framework can be applied to other detector technologies and classification problems where waveform morphology is the distinguishing feature.

In conclusion, the paper argues that $\sqrt{JS}$ distance, potentially with a simple normalization, is the most robust tool for quantifying the dissimilarity between electron and photon event distributions in HPGe detectors, outperforming traditional metrics like Wasserstein or Kolmogorov-Smirnov in stability and sensitivity.