💬 NLP

CANTANTE: Optimizing Agentic Systems via Contrastive Credit Attribution

CANTANTE is a novel framework that optimizes LLM-based multi-agent systems by solving the credit-assignment problem through contrastive attribution of system-level rewards to individual agents, achieving state-of-the-art performance in prompt optimization across diverse benchmarks while reducing inference costs.

Original authors: Tom Zehle

Published 2026-05-14

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Tom Zehle

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a team of three experts working together to solve a difficult puzzle: a Planner who maps out the steps, a Coder who writes the solution, and a Validator who checks if it works. This is what the paper calls a "Multi-Agent System."

The problem the authors, led by Tom Zehle, identified is this: When the team fails, you get a single grade for the whole group (e.g., "You got a C"). But you don't know who messed up. Did the Planner give bad instructions? Did the Coder make a typo? Or did the Validator miss a mistake?

In traditional machine learning, the whole team gets the same grade, and everyone tries to change their behavior based on that single score. This is like a teacher telling a football team, "You lost the game," and then asking the goalkeeper, the striker, and the referee to all change their strategies based on that one sentence. It's inefficient and confusing.

The Solution: CANTANTE

The authors introduce a new framework called CANTANTE. Think of CANTANTE as a super-smart Sports Analyst who watches the team play the same game multiple times with slightly different strategies, and then breaks down exactly who contributed to the win or the loss.

Here is how it works, using a simple analogy:

1. The "What If" Game (Contrastive Attribution)

Instead of just grading the team once, CANTANTE runs the team through the same puzzle three or four times in a row.

Run A: The Planner uses a strict tone, the Coder uses a fast style.
Run B: The Planner uses a friendly tone, the Coder uses the same fast style.
Run C: The Planner uses a strict tone, the Coder uses a slow style.

After running these, the team gets a score for each run. Now, the Attributor (the Analyst) steps in. It looks at the differences:

"In Run A and Run B, the Coder was the same, but the Planner changed. The score went up. So, the Planner's new tone was helpful!"
"In Run A and Run C, the Planner was the same, but the Coder changed. The score went down. So, the Coder's new style was harmful."

This is called Contrastive Attribution. It isolates the contribution of each person by comparing them against each other, rather than just looking at the final score.

2. The "Credit Score" for Each Agent

Once the Analyst figures out who helped and who hurt the team, it gives each agent a specific Credit Score (ranging from -1 to +1).

If the Planner gets a +0.8, the system knows to tweak the Planner's instructions to be more like that successful version.
If the Coder gets a -0.5, the system knows to steer the Coder away from that specific style.

Crucially, the paper claims this is better than just copying the global score. In the old way, if the team failed, the Planner might get blamed even if the Planner did a great job and the Coder was the problem. CANTANTE fixes this by saying, "The Planner did fine; the Coder needs to change."

3. The Results: Smarter Teams, Less Waste

The authors tested this on three very different types of "puzzles":

Coding (MBPP): Writing computer programs.
Math (GSM8K): Solving word problems.
Research (HotpotQA): Answering complex questions that require looking up information in multiple places.

The Findings:

Better Scores: CANTANTE consistently beat other methods. On the coding task, it improved accuracy by nearly 19% compared to the next best method. On the math task, it improved by 12.5%.
Cheaper to Run: Surprisingly, the teams optimized by CANTANTE didn't need to "think" longer or use more computer power to get the right answer. In fact, on the coding and math tasks, they used fewer resources (tokens) than the unoptimized teams.
Stability: The method didn't just get lucky once; it worked consistently across different random starts.

Why This Matters (According to the Paper)

The paper argues that building these AI teams shouldn't be a game of "guess and check" where humans manually tweak prompts. Instead, it should be a science of Credit Assignment.

Just as a coach doesn't yell at the whole team when one player misses a shot, CANTANTE ensures that the AI system learns exactly which part of the team needs to improve. It turns the "black box" of a multi-agent system into a transparent machine where every agent knows exactly how to get better.

In short: CANTANTE is a method that lets AI teams learn from their mistakes by asking, "Who specifically caused this result?" rather than just saying, "We failed, try harder." This leads to smarter, more efficient, and more accurate AI systems.

Technical Summary: CANTANTE

Problem Statement

Large Language Model (LLM)-based multi-agent systems (MAS) have demonstrated strong performance in complex tasks such as software engineering, predictive modeling, and retrieval-augmented generation. However, automating their configuration remains a structural challenge. While system-level evaluation scores (e.g., accuracy, reward) are available, the parameters governing individual agent behavior (such as prompts) are local. This creates a "credit assignment" mismatch: optimizing the system requires converting a global scalar signal into specific update signals for each agent.

Existing approaches often propagate global scores directly to individual agents or update agents sequentially (e.g., round-robin), failing to decompose the reward signal effectively. This results in inefficient optimization where agents may receive incorrect update signals (e.g., a planner being penalized for a coder's error) or where the search space becomes intractable due to the combinatorial nature of joint configurations.

Methodology: CANTANTE

The authors introduce CANTANTE (Contrastive Attribution for Tuning of Multi-Agent Systems), a framework that formalizes MAS optimization as a credit-assignment problem. The core mechanism is contrastive in-group attribution, which decomposes system-level rewards into per-agent update signals.

Core Mechanism

Joint Configuration Sampling: At each iteration, local optimizers propose $K$ candidate parameterizations (e.g., prompts) for each of the $N$ agents. These are assembled into $K$ joint configurations.
Rollout and Evaluation: Each joint configuration is executed on a set of queries, producing system-level scores and execution traces (trajectories of agent outputs).
Contrastive Attribution: Instead of assigning credit based on absolute scores, an attribution model (a prompted LLM) compares the trajectories within a random comparison group of size $g$ $g$ . It assesses how each agent's specific output contributed to the observed differences in system outcomes relative to the other configurations in the group.
- The attributer outputs a scalar credit $c \in [-1, +1]$ for each agent per rollout.
- This allows the system to distinguish between configurations with identical system scores by analyzing intermediate behavioral differences.
Local Updates: The aggregated credits are forwarded to the respective local optimizers, which update the agent parameters (prompts) independently.

Instantiation

In the experiments, CANTANTE is instantiated for prompt optimization, treating agent prompts as learnable system parameters. The framework is agnostic to the choice of local optimizer; the authors utilize CAPO (Cost-aware Prompt Optimization) as the default local optimizer but demonstrate compatibility with others like EvoPrompt.

Key Contributions

Formalization: The paper formalizes LLM-based MAS optimization as a credit-assignment problem, distinguishing it from standard prompt optimization where the system is treated as a monolithic block.
Framework Design: CANTANTE introduces a contrastive attribution mechanism that isolates per-agent contributions by comparing joint rollouts, decoupling attribution from the specific update mechanism.
Empirical Validation: The framework is evaluated against state-of-the-art baselines (GEPA and MIPROv2) across three distinct benchmarks with different workflow topologies:
- MBPP: Programming (conditional workflow with planner, executor, validator).
- GSM8K: Mathematical reasoning (parallel ensembling with three executors and a consensus agent).
- HotpotQA: Multi-hop question answering (retrieval-augmented generation with retriever, reader, synthesizer, and hallucination checker).
Ablation and Analysis: The authors isolate contrastive attribution as the primary driver of performance gains via an "identity attribution" ablation (where global scores are propagated directly). They also analyze credit correlations to diagnose MAS topology bottlenecks.

Results

CANTANTE achieves the best average rank among all evaluated optimizers across the benchmarks.

Performance Gains:
- MBPP: Achieved 41.89% accuracy, an improvement of +18.9 percentage points over the strongest baseline (MIPROv2) and +36.35 points over the initial unoptimized prompts. It also exhibited significantly lower variance across seeds compared to baselines.
- GSM8K: Achieved 82.33% accuracy, improving over MIPROv2 by +12.5 percentage points and over GEPA by 21.06 points.
- HotpotQA: Achieved 11.93% accuracy, remaining within one standard deviation of the strongest baseline and improving over the initial configuration.
Efficiency: On MBPP and GSM8K, CANTANTE achieved higher accuracy while requiring fewer inference-time tokens per query compared to unoptimized prompts and other baselines, indicating that gains were not driven by increased compute but by better parameterization.
Robustness: The method remained effective with reduced dataset sizes (down to 75 samples) and was robust to changes in the attribution model and prompt.
Credit Correlation: Analysis confirmed that the attributer produces meaningful per-agent signals rather than merely echoing global scores. For instance, in GSM8K, later-executing agents received higher attribution due to recency bias, a pattern consistent with known LLM behaviors.

Significance and Claims

The paper claims that CANTANTE represents a step toward treating multi-agent system parameters as objects of machine learning rather than manual configuration. By addressing the structural mismatch between local parameters and global feedback, the framework enables principled adaptation of MAS to new tasks.

The authors emphasize that contrastive attribution is a more effective optimization primitive for LLM-based MAS than global score propagation. The ability to generate meaningful per-agent signals allows for:

More stable and efficient optimization (as seen in lower variance and faster convergence).
Diagnostic capabilities for MAS topology design (identifying which agents are bottlenecks).
Decoupling of the attribution mechanism from the local optimizer, allowing for the integration of sophisticated optimization strategies.

The work is presented as a foundational approach to automating the configuration of complex agentic workflows, reducing the manual labor currently required for deployment.