🤖 AI

Green SARC: Predictive Cost and Carbon Governance for Agentic AI Systems

The paper introduces Green SARC, an architectural governance framework that embeds predictive cost and carbon controls directly into agentic AI loops to prevent budget breaches and achieve significant token, financial, and carbon savings, contrasting its superior reliability against post-execution monitoring and soft penalty methods.

Original authors: Gaston Besanson

Published 2026-06-16

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Gaston Besanson

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you hire a very enthusiastic, highly skilled assistant (an "Agentic AI") to do a complex job for you. This assistant doesn't just do one task; it breaks the job down, uses tools, asks for help from other assistants, and keeps going until the job is done.

The problem? This assistant has no concept of a budget or an energy limit. It might accidentally burn through your entire credit card balance or use enough electricity to power a small city, all while trying to solve a simple problem.

This paper introduces Green SARC, a new "governance system" designed to stop this financial and environmental runaway before it happens. Here is how it works, explained through everyday analogies.

1. The Problem: The "Snowball" and the "Dashboard"

Currently, companies manage AI costs in two ways:

The Dashboard: They look at a bill after the month is over. By then, the money is gone, and the carbon is already emitted. It's like checking your bank account after you've already maxed out your credit card.
The Soft Penalty: They tell the AI, "Please try to be cheap." But the AI is just an optimizer; if it thinks spending more money gets a better result, it will happily overspend. It's like telling a hungry person to "eat moderately" while standing in front of an all-you-can-eat buffet.

The "State Snowball": The paper discovered a specific reason why costs explode. If an AI keeps a running list of everything it has done so far (the "context") and adds it to every new message, the message gets longer and longer.

Analogy: Imagine writing a letter. In the first paragraph, you write 10 words. In the second, you copy the first paragraph and add 10 more words. In the third, you copy the first two and add 10 more. By the 20th paragraph, you are copying thousands of words just to say "Hello."
The paper proves mathematically that this causes costs to grow quadratically (exponentially fast). Real-world data shows this happens even faster than the math predicts.

2. The Solution: The "Pre-Action Gate"

Green SARC installs a Gatekeeper right before the AI takes any action. This isn't a dashboard; it's a bouncer at the door.

How it works: Before the AI sends a message, the Gatekeeper asks: "If you send this, how much will it cost? Do we have enough money left in the budget?"
The Prediction: The Gatekeeper doesn't guess; it uses a learned forecast. It looks at the context and predicts the cost.
The Safety Net: The paper uses a statistical trick called Conformal Prediction. Think of this as a "safety margin." If the AI predicts a cost of $10, the Gatekeeper adds a buffer (say, $5) to be safe. It only lets the action through if $15 is less than the remaining budget.
The Result: The paper shows that this Gatekeeper never lets the budget be breached (0% failure rate in their tests), whereas "soft penalties" (just asking the AI to be careful) fail 91.5% of the time.

3. The Four "Guard Posts"

The system uses four specific checkpoints (borrowed from a framework called SARC) to keep the AI in line:

The Pre-Action Gate (The Bouncer): Stops expensive actions before they happen.
The Action-Time Monitor (The Stopwatch): If the AI gets stuck in a loop (repeating the same mistake over and over), this cuts the power immediately.
The Post-Action Auditor (The Accountant): After the action, it checks: "Did the cost match our prediction?" If the AI lied or the prediction was wrong, it learns from this to get better next time.
The Escalation Router (The Manager): If the budget is totally empty, it hands the task over to a human or a simpler, cheaper system.

4. The Results: Real Savings

The authors tested this on real-world data and synthetic scenarios. Here is what they found:

Massive Savings: By using the Gatekeeper and the other tools, they reduced costs (money, tokens, and carbon) by 47% to 55%.
The "Scope" Knob: The biggest saver was simply capping how much history the AI remembers (stopping the "Snowball"). This is like telling the AI, "Only look at the last 5 minutes of conversation, not the whole day."
The "Routing" Knob: If the AI needs to do a simple task, the system automatically routes it to a cheaper, smaller model. If it needs a hard task, it uses the expensive one. This saves money without changing the number of words used.
Safety First: The system guarantees that the budget is never broken. It's not a "maybe"; it's a hard rule enforced by the architecture.

5. What It Is NOT

It's not a magic wand for quality: The paper admits that if you cut the context too aggressively (to save money), the AI might make mistakes. The system saves money, but the human operator must decide how much quality they are willing to trade for that savings.
It's not just for "Green" goals: While it tracks carbon, the primary driver is financial (FinOps). Saving money and saving carbon happen to go hand-in-hand here.
It's not perfect against liars: If an AI is tricked into thinking a task is cheap when it's actually expensive (an adversarial attack), the Gatekeeper might let it through. However, the "Auditor" will catch this mistake afterward and log it, so the system learns not to be fooled next time.

Summary

Green SARC is a new way to manage AI agents. Instead of waiting for the bill to arrive or hoping the AI behaves, it puts a smart, mathematically proven bouncer at the door. This bouncer predicts the cost, adds a safety buffer, and only lets the AI act if it fits the budget. The result is a system that saves nearly half the money and carbon while guaranteeing you never go over your limit.

Technical Summary: Green SARC

Problem Statement

The cost center of artificial intelligence has shifted from fixed-design training to stochastic inference trajectories in agentic systems. Unlike classical API calls with bounded costs, agentic workflows involve dynamic sequences of model calls, tool invocations, and retries, causing token consumption and associated financial/carbon costs to vary by orders of magnitude even for identical tasks.

Current mitigation strategies—post-hoc auditing and policy-as-code evaluated alongside the agent loop—suffer from a fundamental defect: they evaluate constraints after or beside execution. A budget breach detected at month-end cannot "un-spend" tokens; a carbon overage logged to a dashboard cannot "un-emit" carbon. The paper argues that effective governance requires constraints to be enforced inside the execution loop, predicting costs before actions are committed.

Methodology: Green SARC Architecture

Green SARC applies the SARC (governance-by-architecture) framework to FinOps and GreenOps. It treats cost and carbon constraints as first-class specification objects, enforcing them across four specific sites within the agent loop:

Pre-Action Gate: The core innovation. Instead of a static rule, this site uses a learned, calibrated forecast to predict the token cost and carbon of a proposed action. It admits an action only if the forecast fits the remaining budget with a specified confidence level ( $1-\delta$ ).
Action-Time Monitor: Enforces hard limits on loop counts and marginal costs (circuit breakers) to kill runaway retry or re-plan loops.
Post-Action Auditor: Logs predicted-vs-actual costs to provide an audit trail and generate training signals for the estimator.
Escalation Router: Routes tasks that exhaust budgets or carbon ceilings to human review or deterministic fallbacks.

Theoretical Foundations

State Snowball Theorem: The paper formalizes that naive multi-agent loops, where each step re-submits the full accreted context, incur a cumulative prompt cost of $\Theta(n^2)$ . The leading coefficient is $p/2$ , where $p$ is the per-hop token increment.
Predictive Gate & Calibration: The gate uses an estimator $\hat{f}_\theta$ trained on historical residuals. To ensure safety without assuming a specific residual distribution, the paper employs split-conformal prediction. This provides a distribution-free, finite-sample marginal coverage guarantee: the probability of a budget breach for an admitted action is bounded by $\delta$ .
Anytime-Valid Safety: Using Ville's inequality and sub-Gaussian assumptions (or empirical-Bernstein bounds for heavy-tailed real data), the framework provides a time-uniform bound on cumulative over-spend across any trajectory prefix.
Decoupling: The architecture explicitly decouples cost governance from correctness governance. A system can be safe but expensive, or cheap but unsafe; Green SARC governs the former independently.

Key Contributions

State-Snowball Theorem: A formal proof and empirical confirmation that unconstrained context accretion leads to quadratic cost growth, and that bounding the per-hop increment (via an Adapter Node) collapses this to linear growth.
Calibrated Predictive Gate: A runtime gate that generalizes rule-based accounting to learned forecasts, offering a distribution-free safety guarantee via split-conformal calibration.
Pareto Frontier Dominance: Empirical evidence showing the architectural gate dominates soft-penalty (Lagrangian) approaches. While soft penalties tuned to match a budget in expectation breach that budget on ~91.5% of seeds, the architectural gate breaches on 0% of seeds while maximizing completed work.
Real-Trace Validation: Coverage validation on real ShareGPT data (non-Gaussian residuals) demonstrates that standard Normal- $\sigma$ gates under-cover at tight risk levels, whereas split-conformal calibration maintains nominal coverage.
Open-Source Implementation: A dependency-free Python library (Phase 1) implementing the four-site architecture, including a Normal- $\sigma$ gate and an opt-in split-conformal calibrator.

Experimental Results

Synthetic Evaluation (IBP Workload)

Ablation: Decomposing savings by lever (Scope, Routing, Circuit Breaker).
- Scope (Adapter Node): Drives token reduction (~43-55%) by capping context growth.
- Routing: Converts token savings into USD and carbon savings by switching to efficient models, without further reducing token counts.
- Circuit Breaker: Adds incremental token savings by terminating runaway loops.
Binding Budgets: Under tight budgets (25% of baseline), the gate admits 90% of steps with 0% over-budget incidence, while soft penalties fail to complete trajectories without breaching the budget.
Forecast Quality: The learned estimator reduces Mean Absolute Error (MAE) from ~~4,000 tokens (cold start) to the noise floor (~~34 tokens) within ~20 observations.

Real-Trace Validation

ShareGPT (Chat Traffic): Real conversations exhibit negative cumulative-prompt curvature (concave), meaning the $\Theta(n^2)$ snowball is an artifact of naive orchestration, not inherent to chat. However, the Normal- $\sigma$ gate under-covers on real residuals (skewed, heavy-tailed), while split-conformal holds nominal coverage.
BurstGPT (Production Traffic): On 50,000 real Azure OpenAI requests, the lever ordering (Scope $\to$ Tokens, Routing $\to$ USD/Carbon) holds. The circuit breaker remained dormant (no retry storms), confirming it is a safeguard for multi-step stress scenarios.
SWE-rebench (Multi-Step Agents): On 3,000 real agent plans solving GitHub issues:
- The State Snowball holds: 100% of plans showed positive quadratic curvature ( $\hat{c}_2 > 0$ ), with a median curvature (216) exceeding the linear prediction (134).
- The Circuit Breaker is active: It fired on 11.4% of trajectories, providing the marginal token savings for the full stack.
- Cost-Utility Tradeoff: Aggressive context truncation saves tokens but risks task resolution. A "safe" cap (2 $\times$ median) saved only 2.3% of tokens, while an aggressive cap (0.5 $\times$ ) saved 55% but truncated 100% of plans in the worst-case analysis.

Adversarial Robustness

Scope-aware padding: Attackers can pad prompts to just under the cap to maximize legitimate work; the gate admits these (fundamental limitation of bounded-prompt governance).
Forecast-defeating attacks: Attacks that inflate completion or misreport model costs are caught post-hoc by the Auditor, which logs discrepancies and triggers escalation. The architecture relies on the "predict-act-log-retrain" loop rather than perfect admission-time rejection.

Significance and Claims

The paper claims that Green SARC provides a necessary architectural shift for agentic AI governance. Its significance lies in:

Moving from Post-Hoc to Predictive: Shifting cost control from reconciliation to pre-action prediction with calibrated safety guarantees.
Hard Constraints vs. Soft Penalties: Demonstrating that finite penalties (Lagrangian relaxation) cannot substitute for hard runtime constraints in stochastic environments; only an architectural gate can guarantee budget adherence.
Distribution-Free Safety: Proving that conformal prediction is essential for real-world deployment where residuals are non-Gaussian, ensuring the safety promise holds even when the underlying data distribution shifts.
Modularity: The framework is orthogonal to correctness governance (SARC) and can be deployed independently to manage cloud bills and carbon footprints without altering the agent's core logic.

The authors explicitly note limitations: the current implementation (Phase 1) uses a single-process budget and a Normal- $\sigma$ gate by default (with conformal as an opt-in). The "State Snowball" is confirmed as a failure mode of naive orchestration, not an intrinsic property of conversation. The paper presents these findings with modesty, acknowledging that token savings from scope capping are policy-dependent and trade off against task utility.