🤖 machine learning

On What We Can Learn from Low-Resolution Data

This paper provides a theoretical analysis and empirical evidence demonstrating that incorporating low-resolution data into training sets consistently improves model performance on high-resolution tasks, particularly in resource-constrained domains where high-resolution data is scarce.

Original authors: Theresa Dahl Frehr, Niels Henrik Pontoppidan, Hiba Nassar, Tommy Sonne Alstrøm

Published 2026-05-13

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Theresa Dahl Frehr, Niels Henrik Pontoppidan, Hiba Nassar, Tommy Sonne Alstrøm

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to teach a robot to recognize different animals. Usually, you'd feed it thousands of crystal-clear, high-definition photos. But in the real world, you often can't do that. Maybe the photos are coming from tiny, battery-powered cameras on a smartwatch, or from a hospital system where privacy rules prevent sharing full-quality images. These sources can only send "blurry" or "low-resolution" snapshots.

This paper asks a simple but tricky question: If you have a mix of some sharp photos and many blurry ones, is it worth teaching the robot with the blurry pictures, or should you just throw them away?

The authors say: Don't throw them away! Even blurry pictures help.

Here is how they figured that out, using some creative metaphors:

1. The "Missing Puzzle Pieces" Theory

The researchers looked at what happens when you take a sharp photo and make it blurry. They realized that a blurry photo isn't just a "bad" version of the sharp one; it's a version with specific missing pieces.

Think of a high-resolution image as a complete jigsaw puzzle. A low-resolution image is the same puzzle, but someone has taken out the tiny, intricate pieces that show the fine details (like the texture of fur or the reflection in an eye). The blurry image still has the big, easy-to-see pieces (the shape of the animal, the color of the background).

The paper uses a mathematical tool called Kullback-Leibler (KL) divergence to measure "influence." You can think of this as a "teaching power" score.

The Theory: They proved mathematically that even though the blurry photo is missing pieces, it still teaches the robot something valuable. The "teaching power" of a blurry photo depends on how much information was lost. If the lost information is random noise, the blurry photo is still very helpful. If the lost information is the only thing that matters, then the blurry photo is less useful.
The Analogy: Imagine trying to learn a song. If you have the full, high-quality recording, you hear every instrument. If you only have a low-quality MP3, you might miss the subtle violin notes, but you still hear the main melody and the drums. The paper shows that listening to the "bad MP3" version of a song still helps you learn the tune better than listening to nothing at all.

2. The "Two Types of Learners" Experiment

To test their theory, the researchers trained two different types of AI "students" on a mix of sharp and blurry images:

The CNN (Convolutional Neural Network): This student is like a detective who looks at local clues (edges, textures) and builds a picture piece by piece.
The ViT (Vision Transformer): This student is like a global thinker who looks at the whole picture at once to understand relationships.

They tested these students on standard datasets (like CIFAR-10, which has small pictures of cars, birds, and cats, and AudioMNIST, which has voice recordings).

The Result:
In almost every case, adding the blurry data made the students smarter, especially when they didn't have many sharp photos to begin with.

When the "sharp photo" supply was low, the blurry photos acted as a safety net, boosting the student's performance significantly.
Interestingly, the "Global Thinker" (ViT) seemed to benefit more from the blurry data in image tasks, while the "Detective" (CNN) benefited more in audio tasks. This suggests that different types of AI brains process "blurry" information in different ways.

3. The "Storage vs. Smarts" Trade-off

The paper also looked at the cost. Storing high-resolution data takes up a lot of space (like a heavy suitcase). Storing low-resolution data is light and easy to carry.

They found a sweet spot: You don't need to store everything in high definition.

If you store 90% of your data as "lightweight blurry files" and keep only 10% as "heavy sharp files," your AI model performs almost as well as if you had stored everything in high definition.
This is a huge win for places with limited storage or slow internet (like a rural clinic or a wearable device). You can send the "lightweight" data easily, and the AI still learns effectively.

The Bottom Line

The paper concludes that low-resolution data is not a nuisance; it is a resource.

In a world where we can't always get perfect, high-quality data due to privacy, battery life, or bandwidth limits, we shouldn't just ignore the "blurry" data. By mixing the few sharp examples we have with many blurry ones, we can train smarter, more robust AI systems without needing massive amounts of expensive, high-quality storage.

In short: Don't let the perfect be the enemy of the good. A little bit of high-definition data mixed with a lot of low-definition data is often enough to teach an AI how to see the world.

Technical Summary: On What We Can Learn from Low-Resolution Data

Problem Statement

Modern artificial intelligence systems typically depend on large, centrally collected datasets. However, in real-world domains such as healthcare, public institutions, and distributed ecosystems (e.g., wearable devices like smartwatches and hearing aids), data sharing is often constrained by privacy regulations, storage limitations, bandwidth, and energy resources. Consequently, raw high-resolution data cannot always be transmitted or stored. Instead, devices often transmit feature-based representations or temporally downsampled summaries.

This creates a mixed-resolution setting: a central party (the requester) possesses a limited amount of high-resolution data but must aggregate additional data from external sources that only provide low-resolution samples. The core problem addressed is determining how informative these low-resolution observations are when the ultimate goal is to train models that are evaluated on high-resolution inputs. Existing literature has largely focused on super-resolution (reconstructing high-res from low-res) or improving robustness at inference time, rather than understanding the utility of low-resolution data during the training phase under resource constraints.

Methodology

Theoretical Framework

The authors model the problem using a probabilistic framework where model parameters $\theta$ are treated as random variables following a Gibbs distribution dependent on the training dataset $X$ . They define the influence of a datapoint as the Kullback–Leibler (KL) divergence between the parameter distribution before and after including that datapoint.

Key theoretical steps include:

Decomposition: A high-resolution datapoint $x_h$ is decomposed into a low-resolution component $x_l$ and a residual component $x_r$ (representing information lost during downsampling), such that $x_h = x_l + x_r$ .
KL-Divergence Analysis: The authors derive the ratio of KL-divergences ( $KL_h / KL_l$ ) for including a high-resolution point versus its low-resolution counterpart.
Approximation and Bounds:
- Using a second-order Taylor expansion and cumulant generating functions, they approximate the KL-ratio as the ratio of the variances of the loss function under the posterior distribution: $KL_h / KL_l \approx \text{Var}[\ell(\theta, x_h)] / \text{Var}[\ell(\theta, x_l)]$ .
- They derive bounds relating the relative influence of high- and low-resolution data to the magnitude of the residual component $\|x_r\|$ and the variance of the loss on the low-resolution data.
- The analysis suggests that the benefit of high-resolution data depends not only on the amount of information discarded (the residual) but also on the sensitivity of the loss function to that missing information.

Empirical Evaluation

To validate the theory, the authors conducted experiments on standard benchmarks (CIFAR-10, CIFAR-100, and AudioMNIST) using two distinct architectures:

Vision Transformer (ViT): Handles varying resolutions via patch tokenization and positional embedding interpolation.
Convolutional Neural Network (CNN): Uses adaptive pooling to accommodate different input resolutions.

Experimental Setup:

Scarcity Regime: High-resolution data was limited (e.g., 10% of the dataset), while the remainder was low-resolution.
Comparisons:
- Ratio: Mixed-resolution training vs. training on high-resolution subsets only.
- Size: Mixed-resolution training vs. training on fully downsampled data.
Training Strategy: The authors employed specific loss weighting schedules (cosine annealing) to manage the contribution of low-resolution branches, gradually reducing their weight to focus on high-resolution fine-tuning.

Key Contributions

Theoretical Characterization: The paper provides a theoretical characterization of how the influence of a datapoint changes with resolution. It derives bounds linking the relative contribution of high- and low-resolution observations to the information lost (residual) and the sensitivity of the loss function.
Empirical Validation: Through experiments on vision and audio tasks, the authors demonstrate that adding low-resolution data to a training set consistently improves performance on high-resolution evaluation tasks when high-resolution data is scarce.
Architectural Comparison: The study compares how different inductive biases (CNNs vs. ViTs) interact with downsampling. Results indicate that while both benefit, the magnitude of improvement varies by architecture and modality (e.g., ViTs showed greater gains on CIFAR, while CNNs showed greater gains on AudioMNIST).

Results

Performance Gains: Mixed-resolution training consistently outperformed baselines where only high-resolution subsets were used or where the entire dataset was downsampled. The performance gains were most pronounced when the proportion of high-resolution data was low (e.g., 10%).
Diminishing Returns: As the proportion of high-resolution data increased, the marginal benefit of adding low-resolution data decreased.
Storage Trade-offs: The authors analyzed the storage-cost trade-off. They found that storing a small fraction of data at full resolution (e.g., 10%) combined with the rest at low resolution yielded substantial accuracy improvements compared to fully downsampled datasets, without requiring the storage cost of the full high-resolution dataset.
Theoretical Alignment: Simulations using synthetic data constructed via Discrete Wavelet Transforms (DWT) validated the theoretical bounds. As high-frequency content (the residual) was removed, the influence of low-resolution datapoints decreased, aligning with the derived bounds.

Significance and Claims

The paper argues that low-resolution observations should not be viewed merely as a nuisance to be corrected or ignored, but as a deliberate design choice motivated by practical constraints. The findings suggest that mixed-resolution training is a practical and effective learning regime for settings constrained by storage, bandwidth, or data sharing (such as federated learning or wearable devices).

The authors claim that their work clarifies the role of low-resolution data during training, showing that even when evaluated on high-resolution inputs, models can leverage the task-relevant signal retained in low-resolution samples. This challenges the assumption that high-resolution data is strictly necessary for optimal performance when resources are limited.

Limitations Acknowledged:

The theoretical analysis relies on local linear approximations of the loss, which are most accurate when low-resolution inputs are close to high-resolution ones.
Empirical evaluation was restricted to relatively small-scale benchmarks (CIFAR, AudioMNIST) to allow for controlled variation of data proportions, which may not fully capture the complexity of larger, more diverse real-world datasets.