🤖 AI

Breaking Scale Anchoring: Frequency Representation Learning for Accurate High-Resolution Inference from Low-Resolution Training

This paper introduces the concept of "Scale Anchoring"—where models fail to improve accuracy at higher resolutions due to frequency limitations in low-resolution training data—and proposes Frequency Representation Learning (FRL) to enable more accurate high-resolution spatiotemporal forecasting.

Original authors: Wenshuo Wang, Fan Zhang

Published 2026-02-10

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Wenshuo Wang, Fan Zhang

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Problem: The "Blurry Vision" Trap (Scale Anchoring)

Imagine you are training a professional photographer to take pictures. However, because you are on a budget, you only give them a low-quality, grainy camera from the 1990s. You train them for months, and they become an expert at capturing every detail that the grainy camera can see.

One day, you hand them a brand-new, ultra-high-definition (4K) camera and say, "Go capture the fine textures of a butterfly's wing!"

Even though the new camera is capable of seeing those tiny details, the photographer is stuck. Their brain has been trained to only look for big, blurry shapes. When they see the tiny, sharp details of the butterfly, they don't know how to process them—they might try to "smooth" them out or simply ignore them because, in their experience, "detail" only exists in big, blurry blobs.

In science, this is called Scale Anchoring.

Current AI models used for weather forecasting or fluid simulations (like predicting how smoke moves) suffer from this. We train them on "low-resolution" data (the grainy 90s camera) because high-resolution data is too expensive to produce. When we try to use those models to predict high-resolution reality (the 4K camera), the models fail. They can't "see" the high-frequency details (the tiny ripples in the wind or small swirls in water) because they were never taught that such small things exist.

The Solution: The "Universal Ruler" (Frequency Representation Learning)

The researchers proposed a new method called Frequency Representation Learning (FRL). Instead of just teaching the AI what things look like, they taught it how to scale.

They used three clever tricks to fix the "photographer's" brain:

1. The Multi-Scale Training (The "Zoom" Exercise)

Instead of just showing the AI one type of image, they showed it the same scene at many different zoom levels—from very blurry to medium detail. This teaches the AI that a "swirl" is still a "swirl," whether it's a giant hurricane or a tiny whirlpool.

2. The Normalized Ruler (The "Relative" Secret)

This is the "secret sauce" of the paper. Imagine if you were teaching a child to measure things. If you only ever used a ruler that was 10cm long, they would struggle with a 1-meter table.

The researchers gave the AI a "Normalized Ruler." Instead of telling the AI, "This detail is 0.001 millimeters wide," they taught it to think in percentages: "This detail is 1/100th of the maximum possible detail for this camera." By teaching the AI to think in relative frequencies rather than absolute sizes, the AI learns patterns that work whether the "camera" is low-quality or ultra-HD.

3. The Spectral Consistency Check (The "Fine-Detail" Test)

During training, the researchers added a special "test" to the AI's homework. They didn't just ask, "Does this look right?" They asked, "Does the vibration and texture of this image match the physics of the real world?" This forced the AI to pay attention to the tiny, high-frequency "shivers" in the data that it would normally ignore.

The Result: Breaking the Anchor

When they tested this on complex tasks like 3D fluid simulations (how liquids move) and global weather forecasting, the results were massive:

Old Models: As the resolution got higher, the error stayed the same or even got worse. They were "anchored" to the low-quality training they received.
FRL Models: As the resolution got higher, the models actually got more accurate. They successfully "unlocked" the ability to see the fine details they had never actually seen during training.

In Short...

The researchers didn't just give the AI a better camera; they taught the AI how to understand the concept of "detail" itself, regardless of how blurry or sharp the image happens to be.

Technical Summary: Breaking Scale Anchoring via Frequency Representation Learning

Published at: ICLR 2026
Authors: Wenshuo Wang (SCUT), Fan Zhang (ZJU/MIT)

1. The Problem: Scale Anchoring

The paper identifies a fundamental limitation in Zero-Shot Super-Resolution (ZS-SR) Spatiotemporal Forecasting (STF). Current deep learning models (like Neural Operators) are trained on low-resolution (LR) data and deployed for high-resolution (HR) inference.

While existing research suggests that "successful generalization" means maintaining similar error levels across resolutions, the authors argue this is a misconception. In physics, as resolution increases, the error of a numerical solver should theoretically decrease following a power law ( $\alpha^p$ ). However, current models exhibit Scale Anchoring: their error remains "anchored" to the training resolution.

The Root Cause: The authors define this as an information-theoretic bound. Because the training data is limited by its Nyquist frequency ( $\rho_0/2$ ), the model never "sees" the high-frequency components present in HR inference. Consequently, the model cannot process unseen frequencies, causing the error to be dominated by high-frequency inaccuracies rather than improving with resolution. The authors distinguish this from Spectral Bias (difficulty fitting high frequencies within the training band) and Discretization Mismatch (inconsistency across grids), noting that Scale Anchoring is a hard limit imposed by the data itself.

2. Methodology: Frequency Representation Learning (FRL)

To break this anchor, the authors propose Frequency Representation Learning (FRL), an architecture-agnostic framework designed to decouple the model from the training scale. It consists of three core steps:

Multi-Resolution Data Construction: Instead of training on a single grid, the model is trained on a hierarchy of resolutions created by downsampling the original data. This allows the model to learn relative frequency relationships across different scales.
Nyquist-Normalized Frequency Representation (The Key Innovation): To achieve resolution invariance, the authors introduce a novel positional encoding:
$PE_{freq}(x, k, \rho) = \sin\left(2\pi k \cdot x \cdot \frac{1}{k_{Nyq}(\rho)}\right)$
By normalizing the frequency $k$ relative to the resolution-specific Nyquist frequency ( $k_{Nyq}$ ), the same physical frequency receives an identical embedding regardless of the grid resolution. This allows the network to learn patterns that transfer seamlessly across scales.
Frequency-Aware Training: A spectral consistency loss is added to the standard spatial loss. This forces the model to maintain accuracy in the frequency domain, ensuring that the learned frequency response is stable and consistent across the different training scales.

3. Key Contributions

Identification of Scale Anchoring: A new, fundamental phenomenon in scientific machine learning that explains why ZS-SR models fail to act as true numerical solvers.
Theoretical Framework: Mathematical proofs (Theorems 1 & 2) demonstrating that neural networks are "frequency blind" to components above the training Nyquist limit and that these components dominate the error during HR inference.
FRL Framework: A novel, architecture-agnostic method that uses normalized frequency encodings to enable frequency extrapolation.
Failure Mode Analysis: An investigation into the limits of FRL, noting that it succeeds when physical systems have smooth spectral relationships (e.g., moderate Reynolds number flows) but struggles in highly nonlinear, fully developed turbulence (high Reynolds number).

4. Results and Empirical Validation

The authors tested FRL across diverse architectures (GNN, Transformer, CNN, Diffusion, Neural Operators, Neural ODEs) and two complex domains: 3D Fluid Simulation and Weather Forecasting (ERA5).

Breaking the Anchor: In fluid simulations, baseline models showed an $RMSE_{Ratio} \approx 1$ (error didn't decrease with resolution). FRL-enhanced models reduced this ratio to $0.135–0.375$ , meaning error significantly decreased as resolution increased.
Accuracy Gains: FRL achieved $3.57\times$ to $7.74\times$ improvements in high-resolution RMSE for fluid tasks. In weather forecasting, it improved the Anomaly Correlation Coefficient (ACC) from unreliable levels ( $<0.6$ ) to reliable levels ( $>0.6$ ) at high resolutions.
Efficiency: The method incurs only modest computational overhead (approx. $1.1\times–1.4\times$ training time and $1.3\times–1.5\times$ VRAM) while having negligible impact ( $<2\%$ ) on inference time.

5. Significance

This paper shifts the paradigm of evaluating Zero-Shot Super-Resolution. It moves the goalpost from "maintaining accuracy across scales" to "improving accuracy as resolution increases." By providing a mechanism to extrapolate frequency patterns, FRL allows deep learning models to behave more like traditional numerical solvers, bridging the gap between data-driven learning and physical law consistency. This has profound implications for high-fidelity climate modeling and fluid dynamics where high-resolution data is too expensive to generate for training.