🔢 mathematics

Balancing Inexactness in Mixed Precision Matrix Computations

This paper explores strategies for leveraging mixed-precision arithmetic in high-performance computing by analyzing how to balance numerical errors from low-precision calculations against other inherent sources of inexactness, such as discretization or measurement errors, to enhance performance without compromising overall accuracy.

Original authors: Erin Claire Carson

Published 2026-02-05

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Erin Claire Carson

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Picture: Why "Good Enough" Math is Better

Imagine you are a chef trying to bake a cake for a massive crowd. You have a super-fast, high-tech oven (modern supercomputers) that can bake thousands of cakes a second. However, this oven only works efficiently if you use simple, standard ingredients. If you try to use ultra-precise, rare, and expensive ingredients for every single step, the oven gets bogged down, and you can't bake enough cakes.

In the world of scientific computing, the "ingredients" are numbers, and the "baking" is math.

For decades, scientists insisted on using the most precise numbers possible (called "double precision") for every single calculation, just to be safe. But the author, Erin Carson, argues that this is like using a diamond-tipped knife to cut a piece of bread. It's unnecessary, slow, and wastes the power of modern machines.

The paper suggests a new strategy: Mixed Precision. This means using "rougher," faster, lower-quality numbers for some parts of the math, while saving the super-precise numbers for the critical parts. The goal isn't to be sloppy; it's to be smart about where we spend our "math money."

The Problem: Math is Already "Fuzzy"

The paper points out a funny truth: Scientific math is never perfectly exact to begin with.

Think of a weather forecast.

Modeling Error: The math model of the atmosphere isn't perfect; it's a simplified sketch of reality.
Discretization Error: We can't measure every single drop of rain; we have to guess the weather based on a grid of points.
Rounding Error: Computers can't store infinite decimals, so they have to round numbers off.

Usually, scientists ignore the Rounding Error because they assume the other errors (the sketch and the grid) are so much bigger that the rounding doesn't matter. But now, with super-fast computers that can do math in "low precision" (very rough numbers), the rounding error gets bigger.

If we use low precision blindly, the rounding error might become the biggest problem, ruining our results. The paper's main idea is to balance the errors. We want to make the rounding error just big enough to be fast, but small enough that it doesn't overpower the other "fuzziness" already in our model.

The Toolkit: How Modern Computers Work

The paper looks at modern hardware (like the NVIDIA H100 GPU) that has different "sizes" of number buckets:

Double Precision (64-bit): A huge, detailed bucket. Very accurate, but slow to fill.
Half Precision (16-bit) or FP8: Tiny buckets. They hold less detail and have a smaller range, but they are incredibly fast to fill.

Using the tiny buckets makes the computer run 10 to 50 times faster. But if you try to measure a mountain with a tiny bucket, you might overflow (the bucket spills) or underflow (the bucket is too small to hold the water).

Three Examples of "Balancing the Errors"

The paper gives three specific examples of how to mix these bucket sizes safely:

1. The "Rough Map" Preconditioner

The Scenario: Imagine you are trying to find the shortest path through a massive, tangled maze (solving a linear system). You use a "preconditioner," which is like a rough map to help you navigate.
The Trick: Usually, you draw this map with high detail. The paper shows you can draw the map with a crayon (low precision) instead of a pen.
The Balance: As long as the map is "good enough" to guide you, the fact that the lines are a bit wobbly doesn't matter. The paper proves that if the maze isn't too confusing (ill-conditioned), a crayon map works just fine and saves a huge amount of memory and time.

2. The "One-Pass" Snapshot

The Scenario: Imagine you have a giant photo album (a massive matrix) and you want to find the main themes (eigenvalues). You can only look at the photos once.
The Trick: You take a "snapshot" of the album using a low-resolution camera (low precision) to get the general idea.
The Balance: The paper calculates exactly how blurry the camera can be before the snapshot becomes useless. If the album has very distinct themes (large gaps between eigenvalues), you can use a very blurry camera. If the themes are all very similar, you need a sharper camera. By matching the camera quality to the album's content, you get the result fast without losing the main story.

3. The "Hierarchical" House

The Scenario: Imagine a giant house with many rooms (a hierarchical matrix). Some rooms are empty or simple (off-diagonal blocks), while the main hall is complex (diagonal blocks).
The Trick: You can store the simple rooms in a cardboard box (low precision) and the complex hall in a steel safe (high precision).
The Balance: The paper shows that if the simple rooms are already an approximation (they aren't perfect copies of the real rooms), storing them in a cardboard box doesn't add much new error. You save a massive amount of space, and the total "messiness" of the house stays the same.

The Conclusion: A New Way of Thinking

The paper concludes that we need to stop treating computer rounding errors as the "enemy" to be eliminated at all costs. Instead, we should treat them as just another ingredient in the recipe.

Just as a chef balances salt, sugar, and heat to make a dish, scientists must balance modeling error, discretization error, and rounding error. By carefully choosing which parts of the calculation get the "super-precise" treatment and which parts get the "fast-and-rough" treatment, we can unlock the full speed of modern supercomputers without sacrificing the accuracy of our scientific discoveries.

In short: Don't use a diamond to cut bread. Use the right tool for the job, and you'll get the job done faster.

Technical Summary: Balancing Inexactness in Mixed Precision Matrix Computations

Problem Statement

As computational power enters the "exascale era," high-performance architectures increasingly support arithmetic in multiple precisions and number formats. However, a significant challenge remains: designing algorithms that safely exploit these capabilities without compromising accuracy.

In computational science, matrix computations are inherently inexact due to multiple sources of error, including modeling error, discretization error, linearization error, algorithmic approximation error, and finite-precision rounding error. Historically, these errors have been studied in isolation, with rounding error often ignored under the assumption that double precision (64-bit) is sufficient.

The core problem addressed in this paper is that on modern hardware featuring diverse bit-width formats (e.g., FP16, BF16, FP8), ignoring rounding error and its interaction with other sources of inexactness is both hazardous and a missed opportunity for performance. The paper argues that when other significant sources of inexactness (e.g., low-rank approximations, discretization) are already present, one can safely reduce precision in specific computational steps. The goal is to "balance" these errors—choosing precisions such that rounding error does not dominate the total error budget, thereby improving performance without a noticeable decrease in accuracy.

Methodology

The paper adopts a holistic analytical approach, integrating finite precision error analysis with existing error bounds from other sources of inexactness. Rather than treating rounding error as a separate entity, the methodology involves:

Deriving Mixed Precision Bounds: Establishing theoretical bounds that combine the error from the algorithmic approximation (e.g., truncation, randomization) with the rounding error introduced by lower precision arithmetic.
Error Balancing Criteria: Developing conditions where the finite precision error is bounded relative to the dominant approximation error. This allows for the selection of the lowest possible precision ( $u$ ) that satisfies the condition $u \lesssim \text{approximation error}$ .
Empirical Validation: Conducting numerical experiments using MATLAB 2025a on matrices from the SuiteSparse collection to verify theoretical bounds and demonstrate storage/performance gains.

The paper reviews the history of Mixed Precision Iterative Refinement (IR) as a foundational example of "self-correcting" algorithms where mixed precision is natural, before extending this philosophy to three specific domains where other errors dominate.

Key Contributions and Examples

The paper presents three specific examples demonstrating the "balancing" approach:

1. Mixed Precision Sparse Approximate Inverse (SPAI) Preconditioners

Context: SPAI preconditioners are used in Krylov subspace methods. Constructing them is computationally expensive.
Method: The authors analyze the construction of the sparse approximate inverse $M \approx A^{-1}$ . They derive a condition where the entire construction can be performed in a lower precision $u_s$ provided that $u_s \cdot \text{cond}_2(A) \lesssim \tau$ , where $\tau$ is the stopping tolerance for the residual.
Result: Experiments on the steam1 matrix show that using single precision for the preconditioner construction (within a double-precision GMRES-IR framework) yields a viable alternative to LU preconditioning. While it may require slightly more GMRES iterations, the resulting preconditioner is significantly cheaper to store and apply (e.g., 1,296 non-zeros vs. 14,081 for LU), demonstrating a trade-off between iteration count and application cost.

2. Mixed Precision Randomized Nyström Approximation

Context: This method approximates a symmetric positive semidefinite matrix $A$ using a rank- $k$ Nyström approximation. The dominant cost is the matrix-matrix product $Y = A\Omega$ .
Method: The paper derives a bound for the total error as the sum of the exact approximation error and the finite precision error. A heuristic is proposed: the finite precision error will not dominate if the precision $u_p$ used for accessing $A$ satisfies $u_p \leq n^{-1/2} (\lambda_k / \lambda_1)$ , where $\lambda_k$ and $\lambda_1$ are the $k$ -th and largest eigenvalues.
Result: Experiments on the bcsstm07 matrix confirm that for target ranks $k$ where the eigenvalue decay is slow (high approximation error), half precision can be safely used for the matrix access without degrading the total error compared to exact arithmetic. Single precision is shown to be safe across a wider range.

3. Mixed Precision Hierarchical Matrix (HODLR) Approximations

Context: HODLR matrices approximate off-diagonal blocks as low-rank matrices, reducing complexity from $O(n^2)$ to $O(pn \log n)$ .
Method: Since off-diagonal blocks are already approximated with a relative error $\epsilon$ , the paper analyzes storing these blocks in lower precision $u_k$ . A bound is derived showing that if $u_k \leq \epsilon / (2^{k/2}\xi_k)$ , the global error remains dominated by the approximation error $\epsilon$ , not the rounding error.
Result: An adaptive precision approach is developed. Experiments on various SuiteSparse matrices demonstrate significant storage savings (up to 2x or more) by storing off-diagonal blocks in lower precision (e.g., FP16 or FP8) while maintaining the global error within the target $\epsilon$ . The paper also provides guidelines for setting the working precision in subsequent operations (like matrix-vector products or LU factorization) to ensure backward error does not exceed the representation error.

Results

The numerical experiments consistently demonstrate that:

Performance/Storage Gains: Lower precision can be used for specific components (preconditioner construction, matrix access in randomized methods, off-diagonal blocks in hierarchical matrices) without degrading the final solution accuracy.
Error Dominance: In all cases, the finite precision error remains sub-dominant to the inherent approximation error (discretization, truncation, or low-rank approximation) when the derived balancing conditions are met.
Viability: Mixed precision is not just a theoretical possibility but a practical strategy for reducing memory footprint and increasing throughput on modern exascale hardware.

Significance and Outlook

The paper claims that the significance of this work lies in shifting the paradigm of error analysis. Instead of viewing rounding error as a nuisance to be minimized by using the highest possible precision, it should be viewed as a resource to be balanced against other unavoidable errors.

The authors emphasize that as hardware becomes more heterogeneous (incorporating GPUs, specialized accelerators, and non-IEEE formats), algorithm design must evolve to consider the "whole computational science process." The paper concludes by noting that this balancing act is crucial for future paradigms, including approximate hardware, quantum computing, and neuromorphic computing. The work serves as a call to action for researchers to analyze the specific error sources in their own applications to identify safe opportunities for mixed precision exploitation.

The manuscript is dedicated to Nicholas J. Higham, acknowledging his foundational influence on mixed precision algorithms.