🤖 machine learning

Does Your Neural Network Extrapolate? Feature Engineering as Identifiability Bias for OOD Generalization

This paper argues that out-of-distribution generalization in neural networks depends on "identifiability bias" introduced through feature engineering and structural commitments, which resolve the non-identifiability of data-generating processes from in-distribution data alone, thereby enabling successful extrapolation when the chosen representations correctly capture the underlying causal structure.

Original authors: Leonel Aguilar, Jan Nagler, Christoph Hoelscher, Nino Antulov-Fantulin

Published 2026-05-11

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Leonel Aguilar, Jan Nagler, Christoph Hoelscher, Nino Antulov-Fantulin

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Idea: The "Map vs. Territory" Problem

Imagine you are a tour guide trying to teach a robot how to navigate a city. You only show the robot the neighborhood where you live (the Training Data). You ask the robot to predict what the city looks like in a completely different district it has never visited (the Out-of-Distribution or OOD area).

The paper argues that neural networks are terrible at guessing the unknown unless you give them the right "map" before they start learning.

If you just show the robot raw street names, it might guess the new district looks like a giant, smooth hill because that's the simplest shape it can draw based on your neighborhood. But if the new district is actually a mountain range, the robot will fail miserably.

However, if you give the robot a map that highlights the shape of the terrain (like a topographic map), it can figure out the new district perfectly, even if it's never been there.

The Core Problem: The "Infinite Guessing Game"

The authors prove a mathematical fact: From a single window of data, you cannot know the future.

Imagine two different stories about how the world works:

Story A: The world is a smooth, endless hill.
Story B: The world is a wavy ocean that suddenly turns into a jagged cliff.

If you only look at a tiny patch of land (your training data), both stories might look exactly the same. The hill and the wave look identical in that small patch. Because they look the same here, the robot can't tell which story is true. It picks one, and if it picks the wrong one, it will crash when it hits the cliff.

The paper calls this Observational Equivalence. Without extra help, the data alone cannot tell the robot which story is real.

The Solution: The "Structural Commitment"

So, how do we fix this? The paper says we need to make a Structural Commitment. This is a fancy way of saying: "We are going to force the robot to assume a specific shape for the world."

This commitment has three parts:

The Feature Map ( $\phi$ ): How we translate raw data into a new language (e.g., turning "time" into "sine waves").
The Label Map ( $\psi$ ): How we translate the answers (e.g., turning "population" into "logarithms").
The Model Class ( $M$ ): The type of brain the robot uses (e.g., a simple linear calculator vs. a complex deep thinker).

The Magic Analogy: The Circle Trick
The paper uses a classic example: predicting a sine wave ( $\sin(x)$ ).

The Wrong Way: If you feed the robot raw numbers ( $x$ ), it thinks the world is a straight line or a polynomial curve. When it tries to predict the future, it just keeps going up or down forever. It fails.
The Right Way: If you feed the robot a "Fourier map" (converting $x$ $x$ into a pair of coordinates: $\sin(x)$ $sin (x)$ and $\cos(x)$ $cos (x)$ ), you are essentially telling the robot: "The world is a circle."
- In this new language, the training data and the future data are all just points on the same circle.
- Predicting the future isn't "guessing the unknown" anymore; it's just connecting the dots on a circle (interpolation).
- The robot succeeds perfectly.

What the Experiments Showed

The authors tested this idea in three very different real-world scenarios, and the result was always the same: The right map wins, the wrong map fails.

Chemistry (Mass-Action Kinetics):
- The Task: Predicting how chemicals mix.
- The Result: If you use raw numbers, the prediction goes wild outside the training zone. If you use a "bilinear map" (a specific way of combining the chemical numbers), the robot predicts the future perfectly.
Physics (Kepler's Laws):
- The Task: Predicting how long it takes a planet to orbit a star based on its distance.
- The Result: If you feed the robot raw distance and mass, it fails to predict planets far away. If you tell it to use logarithms (a specific math transformation), it instantly learns the law of gravity and predicts new planets perfectly.
Biology (DNA Detection):
- The Task: Identifying coding DNA in different species (like yeast vs. bacteria).
- The Result: Standard AI models failed when looking at new species. But when the researchers added "biological features" (like counting how often DNA stops or repeats every 3 letters), the model worked across all species, even ones it had never seen.

The "Foundation Model" Twist

The paper also looked at huge, pre-trained AI models (like TimesFM or TabPFN). These are models that have already learned a lot and can't be retrained.

The Finding: Even these smart models fail if you feed them the wrong "language."
The Fix: If you simply change the input format (e.g., instead of giving the model raw numbers, you give it the "log" of the numbers), the frozen model suddenly becomes a genius at extrapolation. You didn't change the brain; you just changed the glasses it was wearing.

The Three Rules for Success

The paper concludes that to make a neural network extrapolate (predict the unknown), you need three things to happen at the same time:

The Right Map: You must transform the data into a format that matches the true shape of the world (e.g., circles for waves, logs for growth).
The Right Brain: The model must be capable of understanding that shape (e.g., a linear model works for circles if the map is right; a complex deep net might fail if it tries to overcomplicate a simple line).
Enough Coverage: You need enough data points in your training window to "tile" the map. If the map is a circle, you need to see enough of the circle to know it's a circle, not just a straight line.

The Bottom Line

Feature Engineering is not dead.

Many people thought that "big data" and "huge models" meant we no longer needed to manually design features or transform data. This paper says: No.

Deep learning is amazing at filling in the blanks inside the data you have. But to guess what happens outside the data, you must explicitly tell the model what the world looks like. You have to provide the "structural commitment." If you don't, the model is just guessing, and it will likely guess wrong.

In short: You can't teach a robot to navigate a new city just by showing it a photo of your street. You have to give it a map that explains the geometry of the city.

Technical Summary: Feature Engineering as Identifiability Bias for OOD Generalization

1. Problem Statement

Deep neural networks excel at interpolation within the support of their training data but frequently fail at out-of-distribution (OOD) extrapolation. While the Universal Approximation Theorem guarantees interpolation capabilities, mathematically guaranteed extrapolation remains an unsolved problem.

The core issue identified in this work is observational equivalence. On any finite training window $W$ , infinitely many data-generating processes (DGPs) can be $\epsilon$ -observationally equivalent (indistinguishable within a tolerance $\epsilon$ ) yet diverge arbitrarily outside $W$ . Consequently, no in-distribution criterion alone (such as training loss, cross-validation, or held-out loss on $W$ ) can reliably distinguish the true underlying process from these confounding hypotheses. The paper argues that OOD extrapolation is non-identifiable from data alone; the choice of representation and model class constitutes a "structural commitment" that resolves this ambiguity.

2. Methodology and Theoretical Framework

2.1 The DGP Commitment $(\phi, \psi, M)$

The authors formalize the learning process as a joint commitment involving three components:

Feature Map ( $\phi$ ): Transforms input $x$ to a representation space $Z_x$ .
Label Map ( $\psi$ ): Transforms the target $y$ to $Z_y$ .
Model Class ( $M$ ): The hypothesis class operating on the transformed spaces.

The paper posits that the triplet $(\phi, \psi, M)$ dictates the assumed DGP. If this commitment aligns with the true DGP such that the target becomes linearly expressible (or otherwise simple) in the transformed space, OOD extrapolation becomes interpolation.

2.2 Observational Equivalence and Identifiability

Proposition 1 establishes that Empirical Risk Minimization (ERM) is blind to observationally equivalent processes. For any finite window $W$ and noise level $\sigma$ , if two processes $P_1$ and $P_2$ agree on $W$ within tolerance $\epsilon$ , no test supported on $W$ can reliably distinguish them when $\epsilon\sqrt{n} \ll \sigma$ . This implies that standard selection criteria (like cross-validation) cannot break the tie between competing DGPs that are observationally equivalent on the training set.

Proposition 2 introduces a heuristic for breaking this tie: near-boundary data. If two processes diverge at a rate $r$ outside the training window, a validation segment of width $\delta \ge \epsilon/r$ immediately adjacent to the training boundary will exit the agreement zone. Scoring candidates on this near-boundary segment allows the selection of the correct structural commitment.

2.3 Constructive Existence

The paper demonstrates that if the true DGP is realizable within the model class $M$ after transformation (i.e., $g^* \circ \phi = f$ on all $X$ , not just $W$ ), and the model is identifiable from finite samples, then OOD error vanishes ( $\epsilon_{OOD} = 0$ ). This is shown for:

Sinusoidal functions: Using Fourier coordinates $\phi(x) = (\sin x, \cos x)$ turns periodic extrapolation into interpolation on a circle $S^1$ .
Power laws: Using log-log transformations.
Exponentials: Using log-y transformations.

3. Key Contributions

Formalization of Observational Equivalence: The paper provides a theoretical lower bound on the ability of any in-distribution test to distinguish between observationally equivalent processes, proving that OOD risk is non-identifiable without structural commitments.
The DGP Commitment Framework: It identifies $(\phi, \psi, M)$ as the sole source of structural information a modeller can supply to resolve OOD ambiguity. It argues that "feature engineering" is not a departure from deep learning success but the explicit injection of the necessary structural commitment.
Controlled Experiments on Minimal Pairs: The authors isolate variables to show that identical architectures with identical in-distribution (ID) loss can differ by orders of magnitude (e.g., $\sim520\times$ ) in OOD performance solely based on the feature map $\phi$ .
Cross-Domain Validation: The mechanism is validated across three natural science domains:
- Chemistry: Mass-action kinetics (MAK) where bilinear features linearize the dynamics.
- Physics: Kepler's Third Law for exoplanet prediction, where log-transforms recover physical exponents.
- Biology: Cross-species coding-DNA detection, where genetic-code-invariant features transfer zero-shot across organisms with varying GC content.
Practitioner Artefacts:
- Near-Boundary Validation: A selection heuristic superior to cross-validation for choosing feature maps.
- $\delta_{OOD}$ Diagnostic: A metric based on SINDy (Sparse Identification of Nonlinear Dynamics) that correlates the residual of a sparse fit on OOD data with the correctness of the feature map.
- Architecture-Invariance Check: A study showing that the OOD guarantee belongs to the feature map (e.g., Exact Fourier Positional Encoding), not the backbone (Transformer, Mamba, or S4D).

4. Experimental Results

4.1 Toy Experiments (Sin(x) and Power Laws)

Sin(x): An MLP trained on raw $x$ extrapolates as a smooth, leveling curve with high OOD error ( $\sim156\%$ ). An identical MLP trained on Fourier features $(\sin x, \cos x)$ achieves near-zero OOD error ( $\sim0.3\%$ ). Foundation models (TimesFM, TabPFN) exhibit the same behavior: they succeed when the input format implicitly provides the correct feature map and fail otherwise.
Power Law vs. Exponential: In a setting where $x^2$ and $e^{\alpha(x-1)}$ are observationally equivalent on $[1, 2]$ , only the correct log-transformed feature map combined with a linear model (OLS) achieves low OOD error. Incorrect commitments (e.g., log-log on an exponential target) result in catastrophic failure ( $>9,999\%$ error).

4.2 Natural Science Domains

Chemistry (MAK): Bilinear features allow OLS to recover the exact governing equations with $0.0\%$ OOD error, whereas raw MLPs fail ( $24.4\%$ ).
Physics (Kepler): Using $(\log a, \log M_{star})$ with OLS recovers the physical law with $3.8\%$ OOD error. Raw MLPs overfit the ID data ( $6.9\%$ ) but fail on OOD ( $79.0\%$ ).
Biology (CDS): Standard positional encodings (learned APE, sinusoidal) fail to generalize across species with different GC contents (dropping to chance levels). Feature maps encoding codon periodicity and amino acid composition enable zero-shot transfer with AUROC up to $0.96$.

4.3 Architecture Invariance (Transformer, Mamba, S4D)

In a periodic sequence task, Exact Fourier Positional Encoding achieves near-zero OOD error ( $\le 0.2\%$ ) across all three architectures and various period lengths. In contrast, standard sinusoidal, RoPE, and learned positional encodings fail ( $>90\%$ error). This confirms the OOD guarantee is a property of the feature map, not the architecture.

4.4 Limits and Necessary Conditions

Using a periodic $d$ -torus benchmark, the paper identifies three jointly necessary conditions for OOD generalization:

Representation Alignment: The feature map must align with the DGP.
Model Class Capacity: The model must be able to express the target's interaction order in the transformed space.
Coverage: The training data must cover the relevant representation space sufficiently (critical coverage $M_{crit} \approx 4$ ).
The study shows that even with a correct feature map, OOD failure occurs if the model class lacks capacity or if sample coverage is insufficient.

5. Significance and Claims

The paper claims to resolve the "when and why" of neural network extrapolation failure. It argues that:

OOD risk is non-identifiable from data on a single window; the structural commitment $(\phi, \psi, M)$ is the only mechanism to resolve this.
Feature engineering is not obsolete. While deep learning can learn features in-distribution, explicit structural commitments are required for extrapolation.
Pretraining does not solve the problem. Pretrained models inherit the commitment problem via their input format; they succeed only when the user supplies the correct transformation.
Correct features are necessary but not sufficient. The model class must express the target, and the data must cover the representation space.

The authors conclude that the "Rashomon set" (the set of models agreeing on training data but disagreeing off-distribution) is navigated not by data alone, but by the modeller's choice of structural commitment. The paper reframes feature engineering as the explicit injection of the DGP commitment required to make extrapolation identifiable.