🤖 AI

Towards Feedback-to-Plan Decisions for Self-Evolving LLM Agents in CUDA Kernel Generation

This paper introduces \texttt{CUDAnalyst}, a unified analysis framework that isolates and attributes the impact of heterogeneous feedback signals on planning decisions in self-evolving LLM agents for CUDA kernel generation, revealing that explicit planning is beneficial only when feedback is aligned and that effective planning emerges from structured multi-feedback interactions.

Original authors: Yee Hin Chong, Jiaming Wu, Youhui Zhang, Peng Qu

Published 2026-05-27

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Yee Hin Chong, Jiaming Wu, Youhui Zhang, Peng Qu

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to teach a robot to write the most efficient code possible for a graphics card (a CUDA kernel). You don't just tell the robot "do it better." Instead, you let the robot write code, run it, and then give it a report card with feedback: "This part is too slow," "This line causes a crash," or "This memory usage is wasteful." The robot then uses this feedback to plan its next attempt.

This paper is about figuring out how the robot actually uses that feedback to make its plans.

The Problem: The "Black Box" of Evolution

In the past, researchers tried to understand this process by simply turning off one type of feedback (like the crash detector) and seeing if the robot got worse. But this is like trying to understand a car engine by driving it for a year, then taking out the spark plugs, and driving it for another year. By the time you compare the two, the car has changed so much from all the other driving that you can't tell if the bad performance was just because of the spark plugs or because the car got tired.

The authors call this "trajectory drift." The robot's path changes so much over time that you can't isolate exactly which piece of feedback helped it make a specific decision.

The Solution: CUDAnalyst (The "Freeze-Frame" Camera)

To fix this, the authors built a tool called CUDAnalyst. Think of it as a time-traveling camera that can freeze the robot's progress at a specific moment.

Freeze the State: They stop the robot's evolution at a specific point in time.
Swap the Feedback: They take that frozen moment and ask the robot to make a plan using only the crash report, then only the speed report, then all of them together.
Compare: Because the starting point (the frozen code) is exactly the same, any difference in the robot's plan is 100% caused by the feedback they changed.

This allows them to see exactly which feedback signals are actually useful and how they work together.

The Big Discoveries (The "Rules of the Road")

Using this freeze-frame method, they found four main things:

1. Feedback is the Fuel; Planning is just the Engine
They found that having a "planning step" (where the robot thinks before acting) is useless unless it has good feedback.

Analogy: Imagine a GPS (the planner) trying to guide a driver. If the GPS has no map data (no feedback), it will just give you random directions, and you'll get lost faster. But if the GPS has real-time traffic data (feedback), it becomes incredibly useful. The paper shows that planning only works when it's grounded in real, aligned feedback.

2. The "Village" Effect (Tools Work Best Together)
The robot uses different tools: a debugger (finds crashes), an analyzer (looks at code structure), and a profiler (measures speed).

Analogy: Think of these tools as a team of doctors. One is a surgeon, one is a radiologist, and one is a nutritionist.
- Early on, the robot needs all of them working together to just get the code to run (survival).
- Later on, the "profiler" (nutritionist) becomes the star for making the code fast, but it still needs the others to make sure the code doesn't break.
- The paper shows that these tools have a "synergy"—they are more powerful together than the sum of their parts.

3. Summaries Help, But Don't Replace the Plan
Sometimes, instead of giving the robot a 50-page report, you give it a 1-page summary.

Analogy: For a smart student (a strong AI model), a 50-page report is fine; they can read it all. But for a student who is still learning (a weaker AI model), a 1-page summary is a huge help because it cuts out the noise.
The Catch: Even with a great summary, the robot still needs a "planner" to decide what to do with that summary. The summary is just the information; the planner is the decision-maker. You can't just hand a summary to the robot and expect it to work perfectly without the planning step.

4. Smart Students Can Teach Dumb Students
The researchers tried taking the "plan" (the strategy) made by a very smart AI and giving it to a weaker AI to follow.

Analogy: It's like a master chess player writing down their strategy notes and giving them to a beginner. The beginner doesn't become a grandmaster instantly, but they play much better than they would on their own.
The Twist: This works best if the two AIs are from the same "family" (trained similarly). If they are too different, the beginner might not understand the master's notes.

The Real-World Result: CuGEdit

Finally, they took these lessons and built a plugin called CuGEdit. This plugin acts like a smart manager for the robot. It knows:

"Right now, the code is broken, so ignore the speed reports and focus on fixing crashes."
"Now that the code works, let's look at the speed reports."
"Let's use the smart AI to write the plan, and the cheaper AI to write the actual code."

When they tested this on a standard benchmark (KernelBench), their system made the code run 2 to 10 times faster than previous methods, proving that understanding how feedback guides planning is the key to building better AI code writers.

Technical Summary: Towards Feedback-to-Plan Decisions for Self-Evolving LLM Agents in CUDA Kernel Generation

Problem Statement

Large Language Models (LLMs) have demonstrated empirical success as self-evolving agents for generating and optimizing CUDA kernels. These systems typically operate via iterative refinement, where heterogeneous feedback signals (e.g., static analysis, runtime profiling, debugging) are aggregated to drive planning decisions for code modification across generations.

However, the mechanism by which individual feedback components influence planning decisions remains opaque. Standard evaluation methods rely on end-to-end (E2E) ablation, where a feedback component is removed, and the entire evolutionary process is restarted. The authors argue this approach is fundamentally flawed for self-evolving agents because:

Trajectory Drift: Iterative planning amplifies early perturbations, causing feedback effects to become conflated with trajectory-dependent drift.
Causal Obscurity: Aggregating non-monotonic, generation-level outcomes into a single scalar obscures when specific signals matter and how they interact.
Lack of Attribution: Practitioners often aggregate diagnostics indiscriminately, lacking utility-aware analysis to determine which feedback components meaningfully influence current planning decisions.

Methodology: CUDAnalyst

To address these limitations, the authors introduce CUDAnalyst, a unified analysis layer designed for controlled, generation-level attribution of planning decisions.

Core Design Principles

Trajectory Freezing: Instead of re-running the evolutionary process, CUDAnalyst freezes intermediate program states at specific generations. This decouples the current planning decision from the historical evolutionary trajectory, ensuring that all interventions share an identical program context.
Selective Feedback Injection: At frozen checkpoints, the system performs controlled patching interventions by selectively withholding or injecting specific feedback components (debugger, analyzer, profiler) while keeping the planner, prompts, and decoding configurations fixed.
Coalitional Attribution: Inspired by cooperative game theory, the authors formalize feedback attribution as a game $G = (N, v)$ . They use the Banzhaf value (rather than Shapley value) to quantify the marginal contribution of each feedback component and the pairwise interaction term ( $\sigma_{ij}$ ) to measure complementarity or redundancy. This allows for fine-grained attribution of how feedback components and their interactions influence planning outcomes.

Evaluation Protocol

The study employs a generation-level feedback intervention protocol:

Fixed Backbone: A reference backbone trajectory is generated (e.g., by a third-party model) and frozen.
Controlled Re-evaluation: Frozen program samples are re-evaluated under different feedback configurations (e.g., full feedback, no feedback, randomized feedback, summarized feedback) without re-running the evolutionary search.
Metrics: Success is measured via execution-level outcomes: compiled (runnable), pass (functional correctness), and fast (performance gain over baseline).

Key Findings

The empirical study, conducted across various model backbones (DeepSeek, Qwen), workloads (PolyBench-ACC, NPB, XSBench, robust-kbench), and reference induction regimes, yields four primary findings:

1. Explicit Planning is Feedback-Conditioned

Explicit planning is not an independent enhancement of intrinsic reasoning capacity. It is effective only when grounded in aligned external feedback.

Planning without feedback (P+NF) consistently degrades performance compared to implicit planning.
Feedback-aligned planning (P+F) yields stable generation-level improvements, particularly for weaker reasoning models.
Counterfactual controls (randomized feedback or dummy plans) confirm that the benefit arises from the semantic alignment of feedback, not merely the presence of planning tokens.

2. Planning Effectiveness Arises from Multi-Tool Interactions

Planning decisions are not dominated by a single feedback source but reflect stable coordination among multiple components.

Analyzer: Consistently supports compilation success.
Profiler: Drives performance gains (fast) in later generations.
Debugger: Exhibits minimal marginal contribution in isolation but is critical for mediating interactions.
Synergy: Interaction terms ( $\sigma_{ij}$ ) strengthen over generations, indicating that later-stage optimization relies on the joint availability of multiple feedback signals to resolve coupled failure modes.

3. Summarization Facilitates but Does Not Replace Planning

Benefit to Weak Models: Summarized feedback significantly improves outcomes for weaker models by reducing representational burden.
Limitation: Summarization cannot replace explicit planning. While summaries compress content, explicit planning maintains and reuses decision structures across generations. Stronger models derive limited additional gains from summarization, suggesting they already possess sufficient capacity to process raw feedback.

4. Cross-Model Plan Transferability

Plans generated by stronger reasoning models can partially transfer to weaker models within the same family.

Injecting plans from strong models (e.g., DeepSeek-R1) into weaker models (e.g., DeepSeek-V3.2) consistently improves execution success.
Gains are largest when models share the same training distribution or representational structure.
This suggests explicit plans act as transferable decision abstractions, functioning as a model-agnostic decision interface rather than a mere reflection of internal reasoning.

Generalization and Robustness

The identified feedback-to-plan structure demonstrates robustness across:

Backbone Trajectories: Tool synergy patterns remain consistent across different underlying code generation models (from weaker open-source to stronger proprietary models), though the focus of synergy shifts from correctness (weaker) to performance optimization (stronger).
Workloads: Qualitative patterns hold across diverse CUDA workloads (NPB, XSBench, deep learning operators), despite variations in feedback noise and kernel complexity.
Reference Induction Regimes: Planning behaviors remain invariant across different evolutionary operators (EoH, MCTS, Hill Climbing), suggesting the feedback-to-plan mechanism is decoupled from specific search dynamics.

Significance and Claims

The paper claims to provide a principled methodology for analyzing self-evolving LLM agents by disentangling feedback attribution from trajectory drift. Key contributions include:

CUDAnalyst Framework: A novel, computationally efficient analysis layer that enables controlled, generation-level interventions and coalitional attribution, overcoming the limitations of E2E ablation.
Structural Insights: The identification of a robust, architecture-invariant "feedback-to-plan" structure, revealing that effective planning is conditional on feedback alignment and multi-tool synergy.
Actionable Design (CuGEdit): The authors operationalize these insights into CuGEdit, a modular plugin that uses similarity-based gating and cross-model plan distillation. Empirical validation on KernelBench Level 3 shows CuGEdit-enhanced agents achieve 2.08× to 10.32× speedup over torch.compile, outperforming baseline and existing SOTA approaches.

The authors conclude that their work establishes a stable framework for credit assignment in self-evolving agents, offering new insights into the internal decision-making dynamics of LLMs in complex, long-horizon code optimization tasks. They emphasize that these findings are specific to the controlled axes studied (frozen trajectories, specific feedback types) and do not necessarily generalize to all agent architectures without further validation.