🤖 machine learning

Explorative Modeling: Unlocking a Third Pretraining Axis and End-to-End Generation

This paper introduces "Explorative Modeling," a new paradigm that factors the training loop to explore multiple generation candidates and select the best, thereby establishing exploration as a scalable third pretraining axis that improves efficiency and performance across domains while enabling true end-to-end generative modeling.

Original authors: Alexi Gladstone, Heng Ji, Yilun Du

Published 2026-07-31

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Alexi Gladstone, Heng Ji, Yilun Du

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to teach a robot to draw. In the old days of machine learning, engineers would build a factory line: one machine drew the outline, another filled in the color, and a third added the eyes. This worked, but it was messy and slow. Then, a revolution happened (sparked by a famous model called AlexNet) that taught us a better way: just give the robot the whole picture and let it learn everything at once, from start to finish. This "end-to-end" approach became the gold standard for almost everything, from recognizing faces to translating languages.

But there was one stubborn problem that refused to play by these new rules: generative AI, the kind of tech that creates new images, videos, or stories. While these models are incredibly talented, they still rely on that old-fashioned factory line. They break the creative process into tiny, tiny steps. For example, to draw a dog, they might first guess a blurry shape, then refine it, then add fur, then add eyes, doing this hundreds of times. The problem is that every time they take a step, they might make a tiny mistake. By the time they finish the hundredth step, those tiny mistakes have piled up, and the dog might end up looking a bit weird or blurry. Scientists have been trying to fix this for years, asking: "Why can't we just train the robot to draw the whole dog in one go, without breaking it into steps?"

The answer, according to a new paper by Alexi Gladstone, Heng Ji, and Yilun Du, is that the world of "what to draw" is messy. A request like "draw a dog" doesn't have just one right answer; there are billions of different dogs. If you try to teach a robot to draw all of them at once without a strategy, it gets confused and just draws a blurry, average-looking dog that looks like nothing real. Existing models solve this by breaking the drawing into steps, but that breaks the "end-to-end" rule. This paper introduces a clever new trick called Explorative Modeling that lets models handle that messiness without needing to break the drawing into steps.

The New Strategy: "Try, Try, and Try Again"

The authors propose a simple but powerful idea: instead of forcing the model to guess the right answer on the very first try, let it try many times and pick the best one. They call this Explorative Modeling.

Imagine you are playing a game where you have to guess a secret number between 1 and 100.

The Old Way (Standard Models): You guess one number. If you are wrong, you get a hint and try again. You keep doing this hundreds of times, getting closer and closer. But every time you guess, you might make a small mistake in how you interpret the hint, and by the end, you might be off by a little bit.
The New Way (Explorative Modeling): You are allowed to shout out 50 different numbers all at once. Then, you look at the secret number and say, "Okay, out of my 50 guesses, number 42 was the closest!" You only learn from that one winner. You ignore the other 49 wrong guesses.

In the paper's language, this is called exploring K candidates. The model generates K different possibilities (like 50 different dogs) and only trains on the one that looks most like the real data. By doing this, the model learns to commit to specific, sharp details (a specific breed of dog) rather than blurring them all together into a generic, fuzzy blob.

Why This Changes Everything

The paper finds that this "try many, pick the best" approach is a game-changer in three major ways:

1. It's a New Superpower for Scaling
Usually, to make AI better, you just make the brain bigger (more parameters) or feed it more data. The authors discovered a third way: you can just make the model "try harder" by increasing the number of guesses it makes during training. They call this exploration.

When they tested this on image, video, and language models, simply increasing the number of guesses made the models significantly better.
The more they scaled up the data and the model size, the more this trick helped. For example, as they added more data, the performance gains from exploration grew from 7% all the way up to 36%. As the models got bigger, the gains jumped from 13% to 23%.
It also made the models much more efficient. They found that using exploration improved the efficiency of the computer's work (FLOPs) by 4.1 times, the efficiency of the data used by 6.2 times, and the efficiency of the model's size by 47%.

2. It Makes Models "End-to-End" Again
Because the model learns by picking the best match from its own guesses, it doesn't need to break the task into tiny steps anymore. It can learn the whole process at once.

The paper shows that with enough exploration, the model can generate a whole image or video in a single step, just like it was trained.
In tests on robotics tasks (teaching a robot arm to move), this new method matched the performance of the best existing methods but used 16 to 256 times fewer computer steps to do it. Instead of taking 100 steps to move a robot arm, the new model did it in just 1 step.

3. It Helps Models Learn Better
The authors suggest that this method helps models generalize better, meaning they don't just memorize the training data but actually learn the rules. When they tested this on video generation, models with more exploration overfit (memorized) less and performed better on new, unseen data. It seems that having the "option" to pick the best match makes the learning process smoother and less confusing.

The Bottom Line

This paper suggests that for over a decade, we've been trying to make AI smarter by making it bigger or feeding it more books. But this research suggests we've been missing a third ingredient: exploration. By letting the AI generate many possibilities and learn from the best one, we can unlock a new level of performance.

The authors are careful to note that this is a new paradigm that is still being explored. They found that while it works incredibly well for images, videos, and some language tasks, it might need more tweaking for other types of models. However, the results are promising: they managed to get an image generator to produce near-perfect results (a score of 1.43 FID on ImageNet) without needing any special "guidance" tricks, and they did it all while making the training process more efficient.

In short, the paper argues that the future of AI might not just be about bigger brains, but about smarter ways of guessing. By giving the AI a chance to try many paths and pick the winner, we can finally teach it to create the whole picture in one go, just like the rest of deep learning has been doing for years.

Technical Summary: Explorative Modeling

1. Problem Statement

The deep learning revolution established that end-to-end training (learning a task directly from data without hand-designed intermediate stages) outperforms decomposed, stage-wise pipelines. However, generative modeling remains a significant exception. Despite the capabilities of modern models like Diffusion and Autoregressive networks, they are not trained end-to-end.

The core issue is that generative modeling involves handling multimodal probability distributions (distributions with many distinct peaks or modes). Existing scalable approaches (e.g., Diffusion, Autoregression) avoid the problem of "mode blurring"—where a model predicts the average of multiple valid modes, resulting in a sample that matches no real data point—by factoring the generation procedure. They break generation into many small, nearly unimodal steps (e.g., denoising steps or token-by-token prediction). While effective, this factorization creates a mismatch between training and inference:

Training: The model predicts a single step conditioned on a specific trajectory or partial data.
Inference: The model is unrolled over hundreds or thousands of steps.
This mismatch leads to exposure bias, where errors compound as the model drifts off the training distribution, preventing true end-to-end generation.

Current scaling laws focus on two axes: parameters (model capacity) and data (training volume). The paper argues these axes miss a third critical capacity: Generative Expressivity (the number of distinct modes a model's training objective allows it to capture). In standard reconstructive models, generative expressivity is fixed by the training objective (often effectively $E=1$ for direct regression), causing performance bottlenecks as models and data scale, regardless of how large they become.

2. Methodology: Explorative Modeling

The authors introduce Explorative Modeling (XMs), a new paradigm that factors the training loop instead of the generation procedure.

Core Mechanism

Instead of forcing a single generation to match a target (which leads to averaging), XMs introduce an exploration step during training:

Candidate Generation: At each training step, the model generates $K$ candidate samples ( $\hat{y}_1, \dots, \hat{y}_K$ ) from the same latent input (e.g., noise).
Selection: The model identifies the candidate closest to the ground truth data $x$ using a reconstruction loss $J$ .
Optimization: The model is updated only on the best match, minimizing the loss over the selected candidate.

Formally, for a data sample $x$ , the objective is:
$L(\theta) = \min_{i \in \{1, \dots, K\}} J(\hat{y}_i, x)$

Variants

The paper defines two directions for exploration:

Forward XM: Fixes the data target $x$ and explores $K$ generated candidates. This is "mass-covering" (prioritizing recall) but computationally expensive as it requires $K$ forward passes per step.
Reverse XM: Fixes a single generated sample $\hat{y}$ and explores $K$ data targets from the dataset. This is "mode-seeking" (prioritizing precision) and computationally cheaper (one forward pass), though it risks mode collapse without entropy constraints.

Theoretical Implications

Generative Expressivity: Exploration increases the model's generative expressivity from 1 to at least $K$ . By selecting the best match among $K$ candidates, the loss minimizer shifts from the blurred mean of the data to the actual data modes.
Substitutability: The paper posits that factoring the training loop (via exploration) is substitutable for factoring the generation procedure. As exploration ( $K$ ) increases, the optimal number of generation steps (jumps) decreases, allowing models to become more end-to-end.

3. Key Contributions

New Paradigm: Introduction of Explorative Modeling, which factors the training loop to handle multimodal distributions, enabling end-to-end generation.
Third Pretraining Axis: Demonstration that exploration is a scalable axis alongside parameters and data. Unlike the other two, it directly scales generative expressivity.
End-to-End Reconstruction: Proof that XMs can function as standalone end-to-end generative models, matching diffusion performance on control tasks with significantly fewer inference steps.
Compute-Generalization Tradeoff: Evidence that spending additional compute on exploration directly improves generalization by reducing overfitting and exposure bias.

4. Experimental Results

The authors evaluated XMs across continuous (images, video) and discrete (language) domains, adding exploration to existing baselines (Diffusion, Flow Matching, Jumpy models, Masked Diffusion Language Models).

Efficiency Gains

FLOP Efficiency: Improved by 4.1× (RAE baseline) and up to 52% (SiT baseline).
Sample Efficiency: Improved by 6.2× (reaching baseline performance with 6.2× less data).
Parameter Efficiency: A Large model with 5 modes explored outperformed an XLarge model with 47% more parameters but no exploration.
Scaling Trends: Efficiency gains increase with scale. When compute was tripled (moving from SiT to RAE settings), efficiency gains more than doubled.

Performance Improvements

Image Generation: XRAE (Explorative RAE) achieved a near-state-of-the-art 1.43 FID on ImageNet (256x256) without guidance, outperforming the guided baselines of previous SOTA recipes.
Video Generation: Increasing exploration monotonically improved FVD scores.
Language: Applying exploration to Masked Diffusion Language Models (XMDLM) improved the Perplexity-Entropy frontier across the board.
Generalization: On video generation tasks, increased exploration reduced overfitting, achieving a better minimum FVD (30.0 vs 37.5 without exploration).

End-to-End Capabilities

Robotics (Behavior Cloning): An Explorative Policy matched Diffusion Policy on Robomimic tasks using 100× less inference compute (1 forward pass vs. 100 steps).
World Modeling: An Explorative World Model matched Diffuser on Maze2D tasks using 16–256× less inference compute.

Scaling Laws

Data Scaling: Performance gains from exploration rose from 7% to 36% as data scale increased.
Model Scaling: Gains rose from 13% to 23% as model size increased.
End-to-End Shift: As exploration increased, models with fewer generation steps (more end-to-end) performed better, confirming that exploration can substitute for generation factorization.

5. Significance and Claims

The paper claims that Explorative Modeling addresses a fundamental bottleneck in generative AI: the inability to scale generative expressivity.

Beyond Likelihood: The authors argue that optimizing likelihood alone is insufficient if the model's expressivity is capped at 1 (predicting the mean). XMs optimize for the ability to capture distinct modes.
True End-to-End Learning: By factoring the training loop, XMs enable models to sample during inference exactly as they do during training, eliminating exposure bias and the train-inference mismatch that plagues current diffusion and autoregressive models.
Scalability: The results suggest that current foundation models, trained without exploration, are increasingly falling short of compute-optimal performance. The paper posits that as models scale, the bottleneck shifts from parameters/data to generative expressivity, making exploration a necessary component for future scaling.
Modest Claims: The authors acknowledge limitations, particularly that Forward XM is computationally expensive for highly multimodal data (like raw images) and that autoregressive LLMs are harder to integrate with exploration due to latent variable injection challenges. They note that Reverse XM may be a more scalable solution for high-multimodality tasks in the future.

In summary, the paper establishes that scaling what models can generate (via exploration) is as critical as scaling what they can represent (parameters) or learn (data), offering a path to fully end-to-end generative modeling.