🤖 AI

Quantifying the Impact of Lossy Compression on Neural Generative Surrogate Modeling

This paper proposes a method to quantify and tolerate compression-induced errors in neural generative surrogate models, demonstrating that lossy compression can reduce storage requirements by up to 39x and training time by 3x while maintaining high model accuracy.

Original authors: Zhimin Li, Harshitha Menon, Charles Jekel, Valerio Pascucci, Peter Lindstrom

Published 2026-06-16

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Zhimin Li, Harshitha Menon, Charles Jekel, Valerio Pascucci, Peter Lindstrom

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to teach a robot chef to recreate a complex, multi-layered cake. The only way to teach it is to show it thousands of photos of the real cake from every angle.

The Problem: The Kitchen is Too Small
The photos of these cakes are incredibly high-definition. They are so large that your kitchen (your computer's storage) is overflowing, and your delivery service (your data transfer speed) is too slow to bring the photos in fast enough. The robot chef is starving for data, but the kitchen can't hold it all.

The Proposed Solution: "Good Enough" Photos
The researchers asked: What if we didn't need perfect photos? What if we could take a slightly blurry, compressed version of the photo—like a JPEG instead of a RAW file—and still teach the robot chef to make a cake that looks and tastes just as good?

This is the core of their work: Can we shrink the training data so much that it fits in the kitchen and arrives faster, without ruining the robot chef's skills?

The Big Discovery: The Chef is Already Imperfect
Here is the clever twist the researchers found. Even if you give the robot chef the perfect high-definition photos, it won't make the exact same cake every time. Why? Because the training process itself is a bit chaotic. The chef might get a random shuffle of the photos, start with a slightly different mood (random weight initialization), or make tiny, unpredictable mistakes.

The researchers realized that the robot chef's natural "imperfections" are actually bigger than the imperfections caused by compressing the photos.

Think of it like this: If you are trying to teach someone to draw a face, and you tell them, "Draw it exactly like this photo," they will still draw it slightly differently every time because their hand shakes a little. If you give them a slightly blurry photo, the difference between the blurry photo and the perfect photo is actually smaller than the difference between the two drawings the artist makes on their own.

The "Safety Net" Method
Instead of guessing how much to compress the data, the researchers built a smart safety net:

They first trained a robot chef using perfect, uncompressed photos.
They measured how much the chef's drawings naturally varied from the real thing.
They then said, "We can compress the photos as much as we want, as long as the 'blur' we introduce is smaller than the natural variation in the chef's drawings."

If the compression error is smaller than the chef's natural "hand shake," the final cake (the model) won't notice the difference.

The Results: A Faster, Leaner Kitchen
They tested this on two complex scientific simulations (think of them as very complicated fluid dynamics recipes, like how oil and water mix in an explosion).

Massive Savings: They were able to shrink the training data by 23.7 times and 39 times. That's like turning a 100-gallon bucket of water into a single cup without losing the ability to cook.
No Quality Loss: The cakes made by the robot chef trained on the tiny, compressed photos looked and behaved exactly the same as those trained on the massive, perfect photos. The physics (like how the "oil" and "water" mixed) were preserved.
Speed Boost: Because the data was so much smaller, it loaded into the kitchen much faster. This sped up the entire training process by 3 times.

The Catch: The Unpacking Cost
There is one small trade-off. When you use compressed data, the computer has to "unpack" or decompress the photos before the chef can look at them.

On slower storage systems, this unpacking was fast enough that the overall speed still improved.
On very fast storage systems, the time spent unpacking sometimes canceled out the speed gains. But in most real-world scenarios, the researchers found that the speed of loading the smaller files won out, making the whole process faster.

In Summary
The paper proves that we don't need perfect data to train powerful AI models for science. By understanding that AI models have their own natural "noise," we can safely throw away a huge amount of data detail. This saves massive amounts of storage space and makes training AI models significantly faster, without sacrificing the quality of the scientific results.

Technical Summary: Quantifying the Impact of Lossy Compression on Neural Generative Surrogate Modeling

Problem Statement

Neural networks are increasingly employed as generative surrogate models to approximate scientific simulations, offering rapid predictions for new input parameters and enabling efficient design optimization. However, training these high-fidelity models requires massive datasets, often reaching terabytes in size, which creates significant burdens on storage, memory, and I/O resources. While lossless compression offers limited reduction, lossy compression techniques (e.g., ZFP, SZ) can achieve substantial data reduction. The central challenge addressed in this work is quantifying the impact of compression-induced errors on the quality of the resulting surrogate model. Current practices often rely on arbitrary compression ratios or conservative settings that miss efficiency gains, or overly aggressive settings that degrade model fidelity. Furthermore, existing evaluation metrics (e.g., single-number prediction accuracy) often fail to capture the nuanced impact of compression on the underlying physical properties and visual fidelity of scientific simulations.

Methodology

The authors propose a novel approach to determine the optimal compression error tolerance by leveraging the inherent variability of neural network training.

1. Characterizing Training Variability

The study establishes that identical training configurations (same data, hyperparameters) can produce different models due to stochastic factors such as random weight initialization, batch shuffling, and stochastic gradient updates. The authors hypothesize that if the error introduced by lossy compression is less than the natural variance introduced by training randomness, the compression can be considered benign.

2. Model-Centric Error Tolerance

Drawing on the Universal Approximation Theorem, the authors argue that neural networks have finite capacity and cannot capture every minute detail in the data. They define two thresholds for compression:

Threshold 1: Precision required to preserve the original data's integrity (often unnecessary for the model).
Threshold 2: The precision level retaining only the features the model is capable of learning.

The core hypothesis is that the model's own prediction error when trained on lossless data serves as a practical upper bound for allowable compression error. If the compression error ( $\epsilon_{loss}$ ) is less than or equal to the model's inherent error on lossless data ( $\epsilon_{model}$ ), the compression will not degrade performance.

3. Algorithm for Adaptive Compression

The authors present an algorithm (Algorithm 1) to determine the compression error tolerance for each sample:

Train a primary surrogate model on lossless data to establish a baseline error distribution ( $\epsilon_{model}$ ).
Iteratively increase the compression error tolerance ( $t_i$ ) for a sample.
Compress the data and measure the resulting L1 compression error.
Continue increasing the tolerance as long as the compression error remains below the baseline model error.
This process avoids the computationally prohibitive cycle of repeatedly compressing data, training, and evaluating.

4. Evaluation Framework

The method is validated using two scientific simulation benchmarks:

Rayleigh-Taylor (RT) Instability: 102,000 samples, 450 GB raw data.
Richtmyer-Meshkov (RM) Instability (PCHIP): 152,184 samples, 893 GB raw data.

Evaluation utilizes:

Quantitative Metrics: Physical conservation laws (total mass, momentum) and mixing layer thickness correlation.
Visual Quality: Peak Signal-to-Noise Ratio (PSNR) distributions.
Performance: Data loading throughput and per-epoch training time across different file systems and GPU configurations.

Key Results

Compression Efficiency and Model Quality

RT Simulation: Training data could be compressed up to 23.7× without significant degradation in physical properties (mass, momentum) or visual quality (PSNR) compared to raw data models. At 32.0× compression, significant deviations in physical quantities and visual artifacts were observed.
PCHIP Simulation: Training data could be compressed up to 39× without noticeably impacting the PSNR distribution or model quality.
Variability Comparison: In both cases, the variance between models trained on raw data (due to training randomness) was comparable to or greater than the variance introduced by compression at the optimal ratios. This confirms that compression errors within these bounds are "benign."

Training Performance

Storage Reduction: Achieved storage savings of 23.7× and 39× for the respective benchmarks.
Training Time: Reducing training data size improved data loading speed, resulting in a total training time reduction of up to 3×.
File System Dependency: The benefits of compression were most pronounced on file systems with lower I/O throughput (File System 1 and 2). On high-performance file systems (File System 3), the overhead of serial decompression sometimes offset the I/O gains, though compression still offered competitive performance. The relative benefit of compression increased with the number of GPUs used, as compute time decreased and I/O became the bottleneck.

Significance and Claims

The paper claims to provide a systematic, non-arbitrary framework for selecting lossy compression ratios for generative surrogate modeling. By grounding the error tolerance in the Universal Approximation Theorem and the inherent variability of neural network training, the authors demonstrate that:

Compression errors can be benign: If compression error is kept below the model's own prediction error (and training variance), the surrogate model maintains high fidelity.
Efficiency gains are substantial: Significant reductions in storage (up to 39×) and training time (up to 3×) are achievable without compromising the scientific utility of the surrogate models.
Evaluation methodology: The study moves beyond simple accuracy metrics to include physics-based conservation laws and visual quality distributions, offering a more robust assessment of surrogate model fidelity.

The authors conclude that their approach allows domain scientists to safely utilize aggressive lossy compression, mitigating the storage and I/O bottlenecks that currently hinder the deployment of neural generative surrogates in large-scale scientific discovery.