🤖 AI

ArchAgent v2: A Case Study with the Data Prefetching Championship

This paper introduces ArchAgent v2, an automated agentic framework that utilizes cascaded evolutionary search and hardware-realizability feedback to successfully design a multi-level data prefetcher that outperforms the winning hand-designed solution in the 4th Data Prefetching Championship.

Original authors: Abraham Gonzalez, Raghav Gupta, Akanksha Jain, Hanna Alam, Alexander Novikov, Po-Sen Huang, Matej Balog, Marvin Eisenberger, Sergey Shirobokov, Ngân Vu, Hank Levy, Borivoje Nikolić, Sagar Karandikar

Published 2026-08-11

📖 3 min read☕ Coffee break read

CC BY 4.0

Original authors: Abraham Gonzalez, Raghav Gupta, Akanksha Jain, Hanna Alam, Alexander Novikov, Po-Sen Huang, Matej Balog, Marvin Eisenberger, Sergey Shirobokov, Ngân V\~u, Hank Levy, Borivoje Nikolić, Sagar Karandikar, Martin Dixon, Parthasarathy Ranganathan

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine a computer's brain as a bustling city where data is the traffic. To keep the city running smoothly, the computer needs to fetch information from its "warehouse" (main memory) and bring it to the "workshop" (the processor) before it's even needed. This is called data prefetching. It's like a smart delivery service that predicts which packages you'll need next and brings them to your door while you're still eating breakfast. However, if the delivery service guesses wrong, it clogs the streets with useless boxes, slowing everyone down. For decades, human engineers have tried to write the perfect delivery algorithm, but the city is so complex, and the rules so strict (like limited space for delivery trucks), that it's incredibly hard to find the best solution. This is where Artificial Intelligence (AI) steps in. Instead of a human trying to guess the perfect route, scientists are now building AI agents that can "evolve" their own solutions, testing thousands of ideas in a virtual city to see which ones actually speed up traffic.

This paper, titled "ArchAgent v2," tells the story of how a team of researchers used a special AI agent to win a high-stakes competition for the best data delivery system. The competition, known as the Fourth Data Prefetching Championship (DPC4), challenged participants to design a system that works across three different levels of computer memory, all while staying within strict size limits. The previous champion was a human-designed system called BertiGO, which was very good but not perfect. The researchers wanted to see if an AI could do better. They introduced a new version of their AI, ArchAgent v2, which uses a clever strategy called "cascaded evolutionary search." Instead of trying to fix all three levels of the memory system at once—which is like trying to redesign the entire city's traffic lights, roads, and delivery trucks simultaneously—the AI tackled them one by one. It first optimized the fastest, most critical level, then moved to the next, and finally the last, before combining them all for a final polish.

The results were impressive. In simulations, ArchAgent v2 automatically designed a new set of prefetching rules that beat the human champion. The AI's solution achieved a 3.8% speedup in performance compared to the standard baseline, and it even nudged past the previous human winner, BertiGO, by 0.3%. The AI was particularly brilliant in "low-bandwidth" scenarios (where the data roads are narrow and crowded), where it improved speed by 4.6%, compared to BertiGO's 2.6%. However, the paper notes that the AI still struggled a bit in multi-core settings (where many processors are working at once), suggesting that while the AI is a powerful tool, the problem of coordinating many processors at once remains a tough challenge.

To understand how the AI did this, the researchers looked at over 12,000 different design ideas the agent generated. They found that the AI didn't just make tiny tweaks; it often made big, structural changes to how the system worked. The winning design succeeded by dynamically adjusting how far ahead it looked for data, slowing down when the roads were too crowded, and smartly deciding which data to keep in the fast workshop versus the slower warehouse. The paper concludes that while this specific AI agent is a major step forward, the journey of automated computer design is just beginning. The AI proved it could synthesize complex logic and beat human experts in a controlled environment, but the path to fully autonomous, perfect computer architecture is still being paved.

Technical Summary: ArchAgent v2 for Multi-Level Data Prefetching

Problem Statement

While agentic artificial intelligence has shown promise in automating algorithm design, scaling these techniques to computer microarchitecture discovery remains difficult. The primary challenges include vast search spaces, strict hardware storage budgets, and long simulation times. Specifically, the original ArchAgent framework successfully discovered single-level cache replacement policies but failed to scale to multi-level data prefetching (L1D, L2, and LLC).

Applying automated discovery to multi-level prefetching introduces three specific architectural difficulties:

Combinatorial Explosion: The design space expands as prefetchers must simultaneously calculate target addresses, optimal timing, and track program/system context.
Non-Monotonic Search Space: Evolving prefetchers at all levels simultaneously creates a complex landscape where a performance-improving mutation at one level can invalidate gains at another.
Hardware Constraints: Prefetchers must adhere to strict, independent physical storage budgets (e.g., 32KB for L1D) that are difficult to enforce in high-level simulators like ChampSim, which typically lack mechanisms to estimate physical area or power costs.

Methodology: ArchAgent v2

To overcome these barriers, the authors introduce ArchAgent v2, a framework that scales automated microarchitecture search to multi-level prefetching through two primary methodological additions:

1. Hardware-Realizability Storage Feedback Loop

To prevent the Large Language Model (LLM) from proposing algorithms that violate physical constraints, the authors implemented an automated interface that calculates the physical storage footprint of each proposed prefetcher.

Mechanism: A custom prefetcher_size() function is integrated into the evaluation loop. It calculates the byte size of the policy based on its state and C++ parameters.
Enforcement: During compilation and simulation, if a candidate policy exceeds the competition's storage budget (32KB L1D, 128KB L2, 256KB LLC), it is immediately rejected and assigned a negative reward.
Prompt Engineering: The LLM prompt was updated (see Figure 3 in the paper) to explicitly instruct the model to update both the logic and the size calculation function synchronously, ensuring size estimates remain accurate as code evolves.

2. Cascaded Evolutionary Search

To manage the complexity of the search space and simulation latency (up to 12 hours per evaluation), the authors replaced simultaneous evolution with a cascaded divide-and-conquer approach.

Sequential Optimization: The framework isolates and optimizes prefetchers at each cache level sequentially (L1D $\to$ $\to$ L2 $\to$ $\to$ LLC).
- First, the L1D prefetcher is evolved (starting from the Berti baseline) while L2 and LLC remain static.
- Once L1D performance plateaus, the best L1D candidate is "frozen," and the L2 prefetcher (starting from Pythia) is evolved.
- Finally, with L1D and L2 frozen, the LLC prefetcher is evolved.
Global Optimization: After individual levels are optimized, the framework spawns global evolutionary runs to optimize all prefetchers simultaneously. These runs utilize a database of successful candidates from individual runs to seed the global search.
Configuration Strategy: Individual level optimizations are performed exclusively on single-core configurations to reduce simulation latency. Multi-core configurations are introduced only during the final global optimization stage. This prioritizes the near-core cache levels (L1D) where latency hiding potential is highest.

Key Contributions

The paper makes four primary contributions:

Cascaded Divide-and-Conquer: A novel evolutionary approach that sequentially evolves subsets of different cache levels (L1D, then L2, then LLC) before performing global optimization.
Automated Storage Feedback: A feedback loop that enforces strict physical storage constraints by calculating and validating prefetcher footprints in real-time during the evolution process.
Performance Validation: Demonstration that ArchAgent v2 discovers a multi-level prefetching policy that outperforms the state-of-the-art (SoTA) hand-designed solution in the Fourth Data Prefetching Championship (DPC4).
Evolutionary Profiling: An analysis of over 12,000 evaluated candidate designs, providing insights into how automated agents explore microarchitectural design spaces, identifying that major breakthroughs often stem from architectural shifts requiring subsequent minor refinements.

Results

Evaluated under the rules of the Fourth Data Prefetching Championship (DPC4), ArchAgent v2 produced a winning policy with the following results:

Overall Performance: Achieved a 3.8% geometric mean IPC speedup over the competition baseline (Berti at L1D, Pythia at L2, no prefetcher at LLC).
Comparison to SoTA: Outperformed the prior championship-winning hand-designed policy, BertiGO, by 0.3% overall.
Low-Bandwidth Single-Core: The policy yielded a 4.6% speedup over the baseline in low-bandwidth single-core configurations, significantly outperforming BertiGO's 2.6% improvement.
Multi-Core Limitations: The policy showed limited gains in multi-core settings compared to single-core, with a slight performance loss (~1.5%) relative to BertiGO in multi-core mixes. The authors attribute this to the cascade prioritizing single-core optimization and the high latency of multi-core simulations impeding the evolution speed.

Design Characteristics of the Winning Policy

L1D: Enhanced the baseline Berti engine with a Global Stride Tracker (GST) and Global Delta History Table (GDHT) to capture cross-IP spatial streams. It introduced a Reinforcement Learning (RL) Arbitrator to select between engines and dynamically throttle bandwidth based on memory signals.
L2: Refined the Pythia baseline by upgrading RL feature representations (expanding from 2 to 5 features) and action spaces (15 to 18 choices), focusing on power-of-two strides and sum-pooling to reduce pollution.
LLC: Added two generators: one for irregular pointer-chasing sequences and one for regular linear/strided patterns. An arbitrator switches between them based on the "chaos" of memory requests.

Significance and Claims

The paper claims that ArchAgent v2 demonstrates automated agentic discovery as a useful tool for computer architects, capable of competing with and surpassing human-derived solutions in complex, constrained environments.

The authors emphasize that the success of the system relies on:

Hardware Realizability: Embedding physical constraints directly into the evolutionary loop is critical for generating viable hardware designs.
Search Decomposition: Breaking down the massive search space into manageable, sequential sub-problems allows the system to navigate the non-monotonic nature of multi-level prefetching.
Synthesis over Novelty: The analysis of the 12,000+ designs suggests that the agent's strength lies in synthesizing and combining existing concepts (e.g., path signatures, velocity monitoring, depth masking) rather than inventing entirely new paradigms. The "winning" design succeeded by dynamically scaling lookahead, throttling bandwidth, and intelligently arbitrating between memory streams.

The paper concludes that while multi-core evolution remains a challenge due to simulation latency, the framework provides a robust methodology for exploring microarchitectural logic, offering a glimpse into how AI-augmented tools can assist in the discovery of complex hardware policies.