🤖 machine learning

Cortex: Compact Behavior Cloning for Quake with Frozen Visual Features

The paper introduces Cortex, a compact behavioral cloning policy for Quake that leverages a frozen DINOv3 visual encoder and a six-layer transformer to achieve consistent level progression and combat success on a large-scale dataset, demonstrating the effectiveness of simple imitation learning before resorting to reinforcement learning or explicit memory.

Original authors: Dzmitry Malyshau

Published 2026-07-28

📖 7 min read🧠 Deep dive

CC BY 4.0

Original authors: Dzmitry Malyshau

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Playground of Digital Minds

Imagine a world where computers don't just follow rigid instructions but learn by watching, much like a child learning to ride a bike by observing their older sibling. This is the realm of machine learning, specifically a branch called behavioral cloning. In this corner of science, researchers feed an AI thousands of hours of human gameplay, hoping the computer can mimic the moves, the timing, and the instincts of a pro player without ever being explicitly told how to play.

The big question driving this field is: How much can a simple, compact AI learn just by copying, before we need to add complex "brain" features like memory, goal-setting, or trial-and-error learning? Think of it as asking how far a parrot can fly just by memorizing the flight path of a hawk, without needing to understand aerodynamics. If we can build a small, efficient AI that plays well just by copying, it saves massive amounts of computing power and energy. But if the AI gets stuck the moment it sees something slightly different from what it practiced, we know we need to teach it how to recover from mistakes. This is the delicate balance between "simple imitation" and "smart adaptation" that scientists are constantly testing.

Cortex: The Tiny Quake Copier

In the paper "Cortex: Compact Behavior Cloning for Quake with Frozen Visual Features," a researcher named Dzmitry Malyshau decided to test the limits of this "simple imitation" idea. He built an AI agent named Cortex and threw it into the chaotic, fast-paced world of the classic 1996 video game Quake. Specifically, he tasked it with navigating the first level, E1M1, a maze of corridors, doors, buttons, and monsters that requires a mix of running, jumping, shooting, and solving simple puzzles.

Cortex is designed to be incredibly lean. It doesn't have a massive brain built from scratch. Instead, it uses a "frozen" visual brain—a pre-trained computer vision model called DINOv3—that acts like a pair of highly trained eyes. These eyes are "frozen," meaning they are already good at seeing shapes and textures and are never changed during training. Cortex's actual "brain" is a tiny, six-layer transformer with only 10.98 million trainable parameters. To put that in perspective, other famous gaming AIs in this space have hundreds of millions of parameters. Cortex is the lightweight sprinter compared to the heavyweight champions.

The training process was surprisingly short and efficient. The AI watched 6,849 recordings of human players, which adds up to about 474.7 hours of gameplay. However, Cortex didn't watch every single second. The researchers selected a specific "sampled epoch," which is like taking a single, carefully curated pass through the data. This involved 517,048 short clips (each four frames long) and took the AI only 3.3 minutes to optimize on a single high-end graphics card (an RTX 5080). The goal was to see if this tiny, fast-trained model could keep up with much larger, more complex AIs.

The Results: A Mixed Bag of Wins and Stumbles

When the researchers let Cortex loose in Quake, the results were a fascinating mix of impressive skill and predictable failure.

The Good News:
Cortex was surprisingly good at the basics. In two separate batches of 20 episodes (each lasting 120 seconds), Cortex managed to reach the "opening door," the "button room," and the "gate descent" in all 20 episodes of each batch. It also managed to get a kill in 19 out of 20 episodes. This suggests that for the early, straightforward parts of the level, simply copying human movements works very well. It could navigate the initial corridors and engage in combat effectively.

The Bad News:
Despite these early wins, Cortex failed to complete the level in any of the 40 episodes (0/20 in both batches). The AI would get stuck, usually near walls or in water, and couldn't figure out how to get unstuck. The researchers noted that this is a classic problem in behavioral cloning called covariate shift: when the AI makes a tiny mistake, it ends up in a situation it has never seen before in the training videos, and since it only knows how to copy what it has seen, it freezes or panics.

The Comparison:
The researchers also tested two other, much larger AI models (P2P-150M and NitroGen) under the same conditions. These giants, trained on thousands of hours of many different games, performed even worse in this specific test. They completed 0 out of 5 episodes each and got stuck almost immediately. This suggests that for this specific, narrow task, a specialized, compact model like Cortex can actually outperform a massive, general-purpose model, at least for the first minute of gameplay.

What the Experiments Revealed

The paper didn't just stop at playing the game; it ran a series of "ablation" experiments to see what made Cortex tick and what broke it.

More Detail Helps Combat, Not Navigation: When the researchers gave Cortex a denser, more detailed view of the screen (using more "tokens" or visual patches), the AI got better at fighting and surviving. It got more kills and died less often. However, this extra detail did not help it navigate the route better. In fact, it sometimes made the route reliability slightly worse. This suggests that seeing more details helps the AI react to enemies, but doesn't necessarily help it understand where to go next.
Memory Isn't a Magic Fix: The team tried giving Cortex a "memory" of its past actions, hoping it would learn from its own mistakes. Surprisingly, this didn't consistently improve its performance. The AI didn't get better at navigating the level just because it remembered what it did five seconds ago.
Offline Metrics Lie: The researchers found that standard computer tests (like how well the AI predicts the next move on a test set) were poor predictors of how well the AI would actually play the game. An AI could look perfect on paper but fail miserably in the actual game. This is a crucial warning for future researchers: don't just trust the numbers; watch the game.

The Verdict

The paper concludes that while a compact, simple AI can learn to play the early parts of a complex game very well by just copying humans, it hits a hard ceiling. It cannot recover from its own mistakes or navigate new situations it hasn't seen before. The failures were consistent with the idea that the AI was suffering from covariate shift—it got lost in the "unknown" parts of the game.

The authors suggest that the next step isn't to make the AI bigger, but to teach it how to recover. They propose a method called DAgger, where a human would step in whenever the AI gets stuck, show it the correct move, and the AI would learn specifically from those "rescue" moments.

In short, Cortex proves that you don't need a supercomputer to play a video game well for a little while. A tiny, efficient model can mimic a human's first few minutes of play with impressive skill. But without a way to learn from its own errors or a memory of what to do when things go wrong, it will eventually hit a wall it can't climb. The paper doesn't claim to have solved the game; instead, it offers a clear, honest map of where simple imitation works and where it breaks down, guiding future researchers on how to build agents that can truly adapt.

Technical Summary: Cortex: Compact Behavior Cloning for Quake with Frozen Visual Features

Problem Statement

The paper investigates the limits of a deliberately simple behavioral cloning (BC) policy in a visually rich, first-person shooter (FPS) environment before introducing reinforcement learning (RL), explicit memory, or goal conditioning. The specific stress test is Quake E1M1, a level requiring a complex sequence of locomotion, door interaction, combat, vertical transitions, and recovery from collisions. The core challenge addressed is the "covariate shift" failure mode common in BC: small errors lead the agent into states absent or rare in the demonstration distribution, causing compounding failures. The authors ask what a compact specialist can learn from ordinary supervised imitation on a single target game without privileged state information (e.g., map, pose, or goals).

Methodology

Architecture

Cortex is a minimal policy designed to map visual inputs directly to actions:

Visual Encoder: A frozen DINOv3 ViT-S+/16 (28.7M parameters) processes 640×400 frames. It outputs a CLS token and a 25×40 patch grid. The policy consumes the CLS token and a uniform 5×8 sample of the patch grid (41 tokens per frame).
Policy Trunk: A six-layer, 384-dimensional, six-head bidirectional transformer. It processes a context of four frames (spanning 300ms), resulting in 164 tokens.
Heads:
- Held-State Head: Predicts 36 independent logits (33 keyboard keys, 3 mouse buttons) using binary cross-entropy.
- Mouse Head: Predicts relative mouse movement (dx/dy) using tanh-squashed values with smooth-L1 loss.
Execution: Held states are sampled independently at temperature 1.0, masked by legal action schemas, and differenced against the previous state to generate press/release events. The policy receives no previous action tokens, pose data, map information, or task text.

Data and Training

Dataset: The Quake subset of the public Pixels2Play (P2P) corpus, containing 6,849 recordings (~474.7 hours) of human keyboard and mouse inputs.
Preprocessing: Source frames (20 fps) are downsampled to 10 Hz. Observations are paired with action intervals starting at $i+1$ .
Training Regimen: The model is trained on a single sampled epoch consisting of 517,048 four-frame windows (3.27% of all valid windows in the training split). This required 32,320 optimizer updates.
Hardware/Time: Training took 3.3 minutes of policy-head optimization on a single RTX 5080 (excluding one-time feature extraction). The evaluated checkpoint is from update 30,000.
Parameters: The system has 10.98M trainable parameters (policy trunk and heads) over the frozen 28.7M parameter encoder, totaling ~39.7M parameters per decision.

Evaluation Protocol

Environment: All systems (Cortex, P2P-150M, NitroGen) run on the same vkQuake build with a read-only engine channel exporting pose, health, and kills.
Time Control: Virtual time is simulated; the game waits while the policy computes to remove stale-frame effects.
Metrics:
- Completion: Reaching the level intermission state.
- Route Waypoints: A heuristic based on the highest index of 15 reference points touched by the player's pose.
- Combat/Survival: Number of kills and time until death.
Batches: Cortex was evaluated in two independent batches of 20 stochastic, 120-second episodes. Reference models (P2P-150M and NitroGen) were evaluated on matched-duration batches of 5 episodes.

Key Results

Primary Evaluation (E1M1 Fresh Spawn)

Completion: Cortex completed the level 0/40 times (across two batches).
Intermediate Progress: Cortex reached pose-based proxies for the opening door, button room, and gate descent in 20/20 episodes in both batches.
Route Performance: The median maximum waypoint index reached was 5 (Batch A) and 6 (Batch B), with a maximum of 9.
Combat: Cortex recorded at least one kill in 19/20 episodes in each batch (Total: 60 kills across 40 episodes).
Survival: 15/20 episodes in Batch A and 18/20 in Batch B ended in death.
Comparison: Under the same time-controlled environment, reference models P2P-150M and NitroGen completed 0/5 episodes each. Their median route scores were 1, and they achieved significantly fewer kills (2 and 0, respectively).

Additional Experiments

Additional Maps (E1M2, E1M3): No system completed these maps. Cortex showed the largest median displacement on E1M2 and more kills, while P2P showed better survival on E1M3.
Mid-Map Starts: When starting from saved states (30s and 60s into a reference trajectory), Cortex showed the largest median displacement from the earlier start, while P2P was more consistent from the later start.
Ablations:
- Visual Density: Increasing patch resolution from 5×8 to full 25×40 improved combat and survival (kills increased, deaths decreased) but slightly reduced route reliability.
- Offline vs. Online: Longer optimization and denser sampling improved offline imitation metrics (held-out loss) but did not consistently improve online route behavior or survival.
- Action History: Adding naive action history tokens did not improve the joint profile of opening, combat, and survival.
- Recovery Sampling: Weighting training windows based on failure-state similarity reduced deaths but introduced new failure modes (corridor sprinting), highlighting a trade-off.

Significance and Claims

The paper positions itself as a modest, rigorous baseline study rather than a demonstration of generalist AI superiority.

Compact Specialist Viability: The authors demonstrate that a compact policy (10.98M trainable params) with frozen visual features can achieve significant intermediate progress (reaching the gate descent) and high combat engagement in a complex FPS level, outperforming larger, multi-game generalist checkpoints in this specific controlled setting.
Limitations of Offline Metrics: The study provides evidence that improved offline imitation metrics (loss, held-out accuracy) do not reliably predict closed-loop online performance (route reliability, survival) in this domain.
Covariate Shift Analysis: The remaining failures (death, getting stuck) are consistent with covariate shift and ambiguous recovery actions, motivating the need for targeted corrective data (e.g., DAgger-style human intervention) rather than more complex architectures.
Reproducibility and Accessibility: The work highlights that a useful closed-loop game-agent research program can be conducted on a single consumer GPU (RTX 5080), provided the heavy lifting of feature extraction and data collection is externalized.
Transparency: The authors explicitly state that the results measure the evaluated releases under a specific harness and do not claim that compact specialists generally outperform foundation-scale agents. They acknowledge that the reference comparisons are limited by small sample sizes, different native interfaces, and adapter choices.

The paper concludes by releasing the compact policy implementation, the exact evaluated weights, and a representative gameplay video, aiming to provide a clear, reproducible baseline for future work in game agents.