📊 statistics

Statistical Taylor Expansion: A New and Path-Independent Method for Uncertainty Analysis

This paper introduces "Statistical Taylor Expansion," a rigorous, path-independent method for uncertainty analysis that replaces precise inputs with random variables to track error propagation through all computational steps, thereby enabling precise quantification of result reliability and exposing significant limitations in conventional numerical approaches and library functions.

Original authors: Chengpu Wang

Published 2026-07-20

📖 9 min read🧠 Deep dive

CC BY 4.0

Original authors: Chengpu Wang

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to measure the height of a growing beanstalk. You grab a ruler, but your hand shakes a little, and the ruler itself has tiny scratches. In the world of science and engineering, we call this "uncertainty." Usually, when we do math with these shaky numbers, we treat them like perfect, solid blocks. We say, "If I multiply 5 by 3, I get 15." But in the real world, if your "5" was actually "5.001" and your "3" was "2.999," the answer isn't exactly 15. It's a messy cloud of possibilities.

For decades, scientists have tried to track this messiness. Some use "interval arithmetic," which is like saying, "The answer is somewhere between 14 and 16." This is safe, but it's often too wide, like trying to find a needle in a haystack by saying, "It's in this whole barn." Others use standard floating-point math, which is fast but ignores the messiness entirely until it causes a crash. The big question is: Can we do math that not only gives us an answer but also tells us exactly how much we can trust it, and how that trust changes as we do more steps? This is the territory of error analysis, a field dedicated to understanding how small mistakes in our measurements grow into big mistakes in our conclusions.

Enter a new method called Statistical Taylor Expansion, proposed by researcher Chengpu Wang. Think of this method as a magical calculator that doesn't just crunch numbers; it crunches distributions. Instead of feeding it a single number like "5," you feed it a "5 with a wobble." The paper argues that by treating every input as a random variable with a known shape (like a bell curve) and a specific number of samples, we can track how that wobble ripples through every single step of a calculation. The most exciting part? This method claims to be path-independent in theory.

To understand why that's a big deal, imagine you are walking from your house to a park. If you take a winding path through the woods, you might get muddy. If you take a straight road, you stay clean. In traditional math, the "mud" (the error) depends entirely on which path you choose. Change the order of your steps, and your final result might be different. Wang's method suggests that if you track the uncertainty statistically, the path you take shouldn't matter in theory. The final "cloud of uncertainty" should be the same, no matter how you got there. However, the paper notes that in practice, accumulated rounding errors can still depend on the order of operations, though the method catches these as additional uncertainty. The paper introduces a practical tool called Variance Arithmetic to test this idea, showing that it can catch hidden errors in standard computer libraries, fix problems in matrix math, and even reveal that some common ways of doing regression (fitting lines to data) are fundamentally flawed because they ignore how variables are connected.

The Core Idea: The Wobbly Ruler

Let's dive into how this works. In the paper, the author starts with a simple concept: a signal isn't just a number; it's a number plus a wobble. If you measure a length as $x \pm \delta x$ , you aren't just saying "it's 5 meters." You are saying, "It's likely 5 meters, but it could be a tiny bit more or less, and here is the shape of that possibility."

The paper builds on a rule called the Uncorrelated Uncertainty Condition. Imagine you have two measurements, like the width and height of a table. If the wobble in the width has nothing to do with the wobble in the height (they are independent), the math is relatively straightforward. The paper uses this to derive a new kind of "Taylor Expansion." You might know the classic Taylor Expansion from calculus, where we approximate a curve using a straight line and then add little corrections. This new version does the same thing, but instead of just adding numbers, it adds variances (the square of the wobble).

The result is a formula that gives you three things for every answer:

The Mean: The most likely answer (like 15).
The Deviation: How wide the cloud of uncertainty is (like $\pm 0.5$ ).
The Reliability: A score from 0 to 1 that tells you how much you can trust that cloud.

The "Path-Independent" Magic

Here is where the paper gets really bold. In standard computer math, the way you write an equation matters. If you calculate $x^2 - x$ , you might get a slightly different answer than if you calculate $x(x-1)$ , even though mathematically they are the same. This is called the Dependency Problem. It's like if you asked two different chefs to bake a cake using the same ingredients but in a different order, and they ended up with slightly different flavors because they lost track of how the ingredients interacted.

Wang's method claims to solve this in theory. By tracking the "wobble" through every intermediate step, the method ensures that the final uncertainty is the same regardless of how you rearranged the math. The paper demonstrates this with Variance Arithmetic, a software implementation that treats every number as a pair: (Value, Uncertainty).

The authors tested this on several tricky scenarios:

Polynomials: They showed that the method can track rounding errors (tiny mistakes computers make when storing numbers) and keep the uncertainty bounds tight.
Matrix Inversion: This is a complex way of solving systems of equations. The paper found that the standard way of doing this (Gaussian elimination) introduces errors because it breaks the dependency between numbers. The new method, which uses a direct formula, keeps the uncertainty honest.
Fast Fourier Transform (FFT): This is a super-fast way to break sound or images into their frequency components. The paper discovered that standard math libraries for sine and cosine functions have hidden numerical errors that can create "resonance" patterns—fake signals that look real. By using a special "Quart Sine" function and Variance Arithmetic, they could spot these errors.

What the Paper Rules Out

The paper is quite critical of some "standard" practices. It explicitly argues against using Interval Arithmetic for random uncertainties. Interval arithmetic is like saying, "The answer is between 14 and 16." The paper shows that this method tends to over-estimate the error, making the range too wide to be useful. It's like saying a car is driving between 0 and 100 mph when you know it's actually going 55 mph. The paper suggests that because most real-world errors are random (like a coin flip) rather than worst-case scenarios, interval arithmetic is the wrong tool.

It also argues that Ordinary Linear Regression (the standard way of fitting a line to data points) is defective. The paper suggests that when you treat data points as having uncertainties, the standard formulas ignore how the errors in the "x" and "y" values are linked. This leads to a "dependency problem" where the calculated line is less accurate than it should be. The paper proposes a "Total Regression" approach that accounts for these links, but it admits this is much harder to calculate and often lacks a simple closed-form solution, forcing a compromise back to the flawed ordinary methods in many cases.

The Findings: How Sure Are We?

The authors have run extensive simulations and tests, and the results are promising but not yet a "solved" problem for every situation.

Successes: In tests involving basic math functions (like sine, cosine, and logarithms) and matrix operations, the method achieved what they call "ideal coverage." This means the calculated uncertainty matched the actual spread of the results almost perfectly (an error deviation of roughly 1.0). For example, when they added noise to a signal and ran it through a Fast Fourier Transform, the method correctly identified the size of the errors.
The "Resonance" Discovery: The paper found that standard computer libraries for sine and cosine functions have small, systematic errors. When these errors interact with certain frequencies, they create a "resonant pattern" that looks like a real signal. The paper suggests that these errors can be significant and that standard floating-point math hides them.
Limitations: The method isn't perfect yet. The authors note that for some complex functions (like logarithms or powers), the math only works if the input uncertainty is small enough. If the "wobble" is too big, the series of corrections diverges, and the method breaks down. They also found that the method requires more computing power than standard math because it has to calculate many more terms.
Confidence Level: The paper presents these findings as measured and simulated. They didn't just guess; they built a tool, ran thousands of tests, and compared the results to known truths. However, they admit that the method is still in "early stages of development." They suggest that while the theory is sound, the practical implementation needs more work to handle all types of probability distributions and to be fast enough for everyday use.

The Takeaway

This paper proposes a new way of thinking about math: Uncertainty is not a bug; it's a feature. By treating every number as a cloud of possibilities and tracking how that cloud changes as we do math, we can get results that are not just numbers, but honest assessments of our knowledge.

The authors suggest that this approach, which they call Statistical Taylor Expansion, could lead to a new branch of mathematics called "Statistical Algebra." It challenges us to rethink how we write algorithms, how we fit lines to data, and how we trust the numbers our computers give us. While it's not a magic wand that fixes everything instantly, it offers a rigorous, theoretically path-independent way to measure the reliability of our calculations, potentially saving us from hidden errors in everything from engineering designs to scientific discoveries. The code is open-source, inviting others to test, improve, and perhaps one day, make this the new standard for how we do math.

Technical Summary: Statistical Taylor Expansion and Variance Arithmetic

1. Problem Statement

Conventional numerical arithmetic faces significant limitations in uncertainty analysis. Standard floating-point arithmetic computes only a precise result value, ignoring rounding errors and input uncertainties. Interval arithmetic, while tracking bounds, suffers from the "dependency problem," where it overestimates result intervals because it treats repeated occurrences of the same variable as independent. Furthermore, traditional numerical algorithms are often "path-dependent," meaning the result and its error characteristics change based on the specific sequence of algebraic operations or algorithmic steps (e.g., catastrophic cancellation).

Existing methods for uncertainty propagation, such as the traditional variance-covariance framework, are generally limited to first-order interactions and assume specific distributional properties. They often fail to account for higher-order interactions, the specific nature of input distributions (e.g., Gaussian vs. Uniform), or the statistical reliability of the result itself. Additionally, standard regression techniques often misuse input uncertainties as weights, violating statistical principles regarding uncorrelated uncertainties.

2. Methodology: Statistical Taylor Expansion

The paper introduces Statistical Taylor Expansion, a rigorous extension of the conventional Taylor series. Instead of treating input variables as precise values, this method replaces them with random variables characterized by a known distribution, a mean, a deviation (uncertainty), and a sample count.

Core Theoretical Framework

Uncorrelated Uncertainty Condition: The method relies on the premise that the uncertainty-level correlation between signals is less than the signal-level correlation when precision is less than 1. This allows the separation of signal and noise components in the propagation of uncertainty.
Higher-Order Expansion: Unlike traditional first-order error propagation, Statistical Taylor Expansion utilizes the full Taylor series to compute the mean and variance of the output.
- The mean of the output $f(x)$ is derived as $f(x) + \sum_{n=2}^{\infty} \frac{(\delta x)^n}{n!} f^{(n)}(x) \zeta(n, \kappa)$ , where $\zeta(n, \kappa)$ represents bound moments of the normalized input distribution.
- The variance $\delta^2 f(x)$ is computed by summing higher-order interaction terms involving derivatives and bound moments, capturing non-linear effects and higher-order interactions between variables.
Path Independence: By tracking dependencies through the complete analytic form of a function rather than decomposed arithmetic steps, the method ensures that the calculated mean and variance are theoretically path-independent. This contrasts with conventional numerical methods where the order of operations affects the accumulation of rounding errors and dependency artifacts.
Statistical Bounding: The method introduces a "bounding leakage" parameter ( $\epsilon$ ) and a bound range ( $\kappa$ ) to define the reliability of the variance estimate. It distinguishes between "distributional poles" (where the derivative is zero, causing density to spike) and "distributional zeros" (where density vanishes), analyzing convergence conditions for functions like $\log(x)$ and $x^c$ .

Implementation: Variance Arithmetic

The paper presents Variance Arithmetic as a numerical implementation of Statistical Taylor Expansion. It represents an imprecise value $x \pm \delta x$ as a pair of 64-bit floating-point numbers. To ensure numerical stability and validity, the implementation enforces specific rules:

Finite: Results and variances must remain finite.
Monotonic: The expansion terms must decrease monotonically in absolute value to ensure convergence.
Stable: The truncation error must be below a threshold defined by the bounding leakage.
Positive: Variance must remain positive at every expansion order.
Reliable: The deviation of the variance must be within a specific tolerance.

The method also incorporates floating-point rounding errors as part of the input deviation $\delta x$ , treating them as uniformly distributed noise.

3. Key Contributions and Results

Theoretical Advancements

Uncertainty Bias: The study defines "uncertainty bias" as the difference between the true mean of the output and the function evaluated at the input mean, quantifying how input uncertainty shifts the expected result.
Convergence Analysis: The paper establishes mathematical conditions for the convergence of the variance series, particularly for functions with distributional poles or zeros (e.g., $\log(x)$ and $x^c$ ). It demonstrates that convergence depends on the input precision relative to the bound range $\kappa$ .
Statistical Algebra: The paper proposes "Statistical Algebra" as a new framework for solving equations with random variables, extending analytic solutions to stochastic contexts (e.g., differential equations with random coefficients).

Empirical Demonstrations

The paper validates Variance Arithmetic across diverse applications:

Polynomial Computation: Demonstrates the ability to trace floating-point rounding errors and maintain continuity in parameter space, even across distributional poles.
Matrix Inversion: Shows that using the analytic definition of the inverse (via the adjugate matrix) rather than Gaussian elimination avoids the dependency problem. It reveals a strong linear correlation between matrix condition numbers and determinant precision, suggesting that uncertainty can be a better metric for matrix stability than condition numbers alone.
Mathematical Library Functions: Evaluates standard functions ( $e^x, \log x, \sin x$ ). It reveals that standard library functions (e.g., sin, cos) contain significant numerical errors that can cause "resonance" patterns in Fast Fourier Transform (FFT) results, leading to large value errors that conventional arithmetic fails to detect.
FFT Analysis: In FFT applications, the method detects that using standard library sine functions introduces systematic errors that grow with the FFT order. By using a "Quart sine" function (based on integer indices) to eliminate floating-point rounding in the input, the method achieves ideal coverage. It shows that interval arithmetic significantly overestimates uncertainty in these random contexts, whereas Variance Arithmetic tracks the actual error distribution with high precision.
Regression: The paper critiques conventional least-squares regression for treating input uncertainties as weights. It proposes "Total Regression" and "Partition and Weight" methods that rigorously handle within-subset and between-subset covariances, avoiding the dependency problems inherent in ordinary multi-dimensional regression.
Hydrogen Spectra: Applied to the calculation of the fine-structure constant ( $\alpha$ ) from hydrogen spectral lines, the method quantifies the impact of measurement uncertainties and identifies that the variance between different measurement subsets accounts for only 14% of the total variance, suggesting specific avenues for experimental improvement.

Validation Metrics

The paper defines two primary validation tests:

Distribution Test: Compares the calculated mean and deviation against a dataset of results. Ideal coverage yields an error deviation of exactly 1 and a Normal error distribution.
Value Test: Compares calculated values against known precise results (e.g., round-trip transformations). Ideal coverage yields an error distribution concentrated at zero (Delta-like) with an error slope of -1.

4. Significance and Claims

The paper claims that Statistical Taylor Expansion fundamentally alters the approach to numerical uncertainty analysis by rendering results path-independent. This is a distinct departure from conventional numerical approaches where the result depends on the specific algebraic form or algorithmic path taken.

Key claims regarding significance include:

Revealing Hidden Errors: The method can detect substantial numerical errors in standard mathematical library functions and floating-point rounding errors that are invisible to conventional arithmetic or overestimated by interval arithmetic.
Quantifying Reliability: It provides a "bounding reliability" (a value between 0 and 1) for results, derived from input sample counts and distributions, which is a novel capability.
Redefining Regression: It exposes defects in conventional regression practices, specifically the misuse of input uncertainties as weights, and offers a statistically rigorous alternative.
New Mathematical Paradigm: The introduction of "Statistical Algebra" suggests a pathway for solving equations with random variables directly, rather than relying on sampling-based stochastic methods.

The authors acknowledge that the method reduces execution flexibility (due to the requirement for complete analytic forms) and that the current implementation (Variance Arithmetic) is in early development. They note that while the method handles random uncertainties effectively, challenges remain in adapting it to scenarios lacking closed-form analytic solutions or explicit input uncertainties, such as general floating-point rounding error tracking without hardware support.

The study concludes that for applications where uncertainties are random in nature (the majority of scientific and engineering contexts), Statistical Taylor Expansion offers a more precise and suitable framework for uncertainty characterization than interval arithmetic.