🤖 AI

SAGE: Safety-First Defense-in-Depth Guardrails for Verified Lifecycle Control of High-Impact Generative AI

The paper introduces SAGE, a safety-first, authorization-separated architecture for high-impact generative AI that prioritizes catastrophic risk mitigation through formal verification and a comprehensive defense-in-depth framework, validated by a multi-model study demonstrating low harmful-compliance rates and specific performance contrasts among leading AI snapshots.

Original authors: Mahdi Eslamimehr

Published 2026-07-28

📖 7 min read🧠 Deep dive

CC BY 4.0

Original authors: Mahdi Eslamimehr

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Safety Net Before the Jump

Imagine you are building a robot that can write code, give medical advice, or explain complex science. This is the world of Generative AI, a technology that creates new content rather than just searching for old answers. But here's the catch: if this robot gets a little too creative, it might accidentally help someone build a bomb, hack a bank, or spread dangerous lies. This isn't just about the robot being "rude"; it's about preventing catastrophic harm.

To stop this, scientists use Guardrails. Think of these as the rules of the road for the robot. Usually, guardrails are like a bouncer at a club who checks your ID at the door. If you look suspicious, you don't get in. But what if the bouncer misses something? What if the robot finds a back door? This is where the idea of Defense-in-Depth comes in. Instead of relying on just one bouncer, you have a fence, a moat, a guard dog, and a second bouncer inside. You layer your safety so that if one thing fails, the next one catches the problem.

The big question everyone is asking is: Which robot is actually the safest? Is the one that says "No" the most often the best, or is it the one that says "No" to bad things but still helps with good things? We need a way to test these robots not just on how well they follow rules, but on how they handle the whole journey from being built to being used, ensuring that safety is the most important thing, even more than speed or profit.

The SAGE System: A Safety-First Super-Structure

Enter SAGE (Safety-First Defense-in-Depth Guardrails for Verified Lifecycle Control). Think of SAGE not just as a list of rules, but as a high-tech, unbreakable safety vault for AI. The authors, researchers at Quandary Peak Research, argue that safety shouldn't be an afterthought or a simple filter. Instead, it needs to be the boss.

Imagine you are running a very serious science experiment. Before you even turn on the machine, you have to sign a Release Manifest. This is like a magical, tamper-proof contract that says, "We have checked this machine, we have tested it against bad guys, and we promise it won't explode." If the machine changes even a tiny bit, the contract breaks, and the machine shuts down. SAGE uses these signed manifests to make sure that only safe versions of AI are ever allowed to run.

Once the AI is running, SAGE acts like a traffic cop with a superpower. It doesn't just look at the question you ask; it looks at the risk of the answer.

The Safety Gate: If the AI thinks an answer might help someone do something terrible (like a cyberattack or a chemical weapon), it slams the gate shut. It doesn't matter if the answer would be super useful or fast; if it's dangerous, it's a hard "No."
The Safety Net: If the AI is unsure, it doesn't guess. It switches to a "safe mode," giving a boring but safe answer or asking for a human to check it first.
The Time Machine: If something goes wrong while the AI is working, SAGE has a Rollback button. It's like a video game "undo" feature that instantly reverts the system to a safe, known state before the mistake happened.

The Great Robot Race: What They Found

To see how well this system works in the real world, the researchers set up a massive, frozen test. They didn't just ask the robots one question; they sent 84 specific, tricky cases to 10 different AI snapshots (versions of models from OpenAI, Anthropic, and Google). They treated every robot exactly the same, like a fair race where everyone runs on the same track at the same time.

Here is what they discovered:

The "No" isn't the only thing that matters: Many people think the safest AI is the one that refuses to answer everything. But the study found that the "safest" robots were actually the ones that could say "No" to bad requests and still say "Yes" to helpful, harmless ones.
The Winners: The top performers were Claude Haiku 4.5, Claude Sonnet 4.6, and two versions of Gemini. These models were incredibly good at spotting danger and refusing it, but they were also great at being helpful with normal questions. They got a "Balanced Guardrail Score" (a score that mixes safety and helpfulness) of nearly perfect, around 0.99 to 1.00.
The Strugglers: The GPT-5 mini and GPT-5 nano models were still very safe (they rarely gave dangerous answers), but they were a bit more clumsy. They refused to answer harmless questions more often, and when they tried to give safe alternatives, they weren't as good at it. Their scores were lower, around 0.84 to 0.86.
The Surprise: The biggest difference between the winners and the losers wasn't that the losers were dangerous. The losers were actually quite safe! The difference was that the winners were more useful and better at redirecting people to safe topics.

What the Paper Says (and Doesn't Say)

The researchers are very careful not to declare a total victory. They found that while some robots were clearly better than others in this specific test, the differences weren't huge when it came to actually preventing harm. In fact, for the most dangerous questions, almost all the robots did a good job saying "No."

However, the paper explicitly rules out a few things:

It's not a final ranking: You can't say "OpenAI is the worst" or "Anthropic is the best" forever. These are just snapshots of specific versions on a specific day.
It's not a guarantee: The study only asked one question per robot. In the real world, bad actors might try thousands of tricks. The paper admits that the "harmful compliance" (how often the robot actually did something bad) was very low in their test, but that doesn't mean it's impossible for a robot to fail in a more complex, real-world scenario.
It's not a magic shield: The SAGE system is a blueprint. It's a set of instructions on how to build a safe AI, but the paper didn't actually deploy this system to run a real company. It's a design, not a finished product.

The Bottom Line

The main takeaway is that safety isn't just about being a grumpy robot that says "No" to everything. The best safety system is a layered fortress that checks the AI before it starts, watches it while it works, and has a plan to fix things if it slips up.

The study suggests that the best AI models are the ones that can be smart and helpful without being dangerous. The robots that scored highest were the ones that could tell the difference between a bad request and a good one, refusing the bad ones firmly but still being friendly and useful for the good ones. The paper concludes that we need to keep testing, keep layering our safety nets, and never assume that just because a robot passed one test, it's ready for the real world. Safety is a journey, not a destination.

Technical Summary: SAGE – Safety-First Defense-in-Depth Guardrails for Verified Lifecycle Control of High-Impact Generative AI

Problem Statement
The paper addresses the critical challenge of preventing catastrophic misuse of high-impact generative AI (e.g., terrorism, cyberattacks, biological threats). It argues that safety is not merely a prompt-filtering issue but a lifecycle-control problem. Current approaches often treat safety as a trade-off against utility or latency. The authors posit that for high-stakes applications, the prevention of "catastrophic enablement" must be a hard constraint that precedes all other objectives. Furthermore, safety mechanisms must avoid conflating content risk with user intent, ensuring that conservative response controls do not automatically trigger surveillance or external reporting without independent human authorization.

Methodology: The SAGE Architecture
The paper proposes SAGE, a safety-first, authorization-separated architecture designed to constrain admissibility based on credible catastrophic-enablement risk. The framework integrates formal verification, runtime monitoring, and a signed release manifest.

Formal Framework and Risk Modeling:
- Action Space: The system defines a lexicographic action set $A = \{G_0, G_1, G_2, G_3, G_4\}$ , ranging from normal answers to internal review and authorized incident processes.
- Robust Risk ( $K$ ): The system calculates a robust primary risk $K(g|e)$ using an ambiguity set $\Pi(e)$ of posteriors. This accounts for calibrated evidence and declared distribution shift tolerances ( $\epsilon_e$ ). The policy selects the action that minimizes secondary loss (utility, privacy, delay) only within the set of actions where $K(g|e)$ is below a safety ceiling $\kappa(e)$ .
- Defense-in-Depth Bounds: The paper derives formal bounds for detector miss rates. It establishes that under independent detectors, failure probability is the product of individual misses. Crucially, it introduces a common-cause dependence bound (Proposition 4), showing that if a common-mode degradation event exists with probability $\gamma$ , the joint miss probability cannot be certified below $\gamma$ , regardless of the number of layers. This necessitates diversity in detectors, training data, and policy dependencies.
- Authorization Cut: A formal theorem (Theorem 4) proves that unprivileged components (e.g., classifiers) cannot reach authorized states (e.g., external disclosure) if the state transition graph is structurally constrained, preventing classifiers from "minting" exceptional authority.
Lifecycle Control and Release Manifests:
- Signed Manifests: Every release candidate is bound to a signed manifest containing model hashes, risk thresholds, readiness bits, and expiry dates.
- Monotone Gating: The release predicate is monotone; tightening risk ceilings or adding blocking findings can only render a release ineligible, never eligible.
- Runtime Enforcement: At runtime, the system uses authenticated ingress, diverse detectors, and output checking. If a risk threshold is breached, the system can restrict service, disable tools, invoke safe fallbacks, or roll back to a previous approved artifact.
- Three-Valued Monitoring: A runtime monitor evaluates temporal properties as True, False, or Unknown (?), ensuring that decisions are only made when infinite continuations of a trace are definitively safe or unsafe.
Empirical Evaluation Protocol:
- Frozen Protocol: The study used a frozen, vendor-symmetric protocol to test 10 model snapshots (OpenAI GPT-5 series, Anthropic Claude 4.5/4.6/4.7 series, Google Gemini 3/3.1 series).
- Dataset: 840 calls were made (84 per snapshot) covering 375 unique responses after filtering for provider errors and incomplete judgments. The dataset included direct unsafe requests, benign sensitive-looking prompts, and robustness wrappers.
- Metrics: The study measured Harmful Compliance Score (HCS), Benign Utility (BU), Redirection Delivery (RD), and Over-Refusal Rate (ORR). These were combined into a Balanced Guardrail Score (BGS).
- Judging: Responses were judged by blinded, non-target-family models using a strict rubric. The study explicitly avoided human adjudication to maintain protocol consistency, acknowledging this as a limitation.

Key Results

Harmful Compliance: Observed harmful-compliance estimates were low across all models, ranging from 0.0000 to 0.0333.
Differentiation Drivers: The primary separation between models was not in harmful compliance (which was uniformly low) but in benign utility and safe redirection.
- Top performers (e.g., Claude Haiku 4.5, Claude Sonnet 4.6, Gemini 3 Flash) achieved near-perfect redirection and high utility.
- Lower-scoring models (e.g., GPT-5 mini, GPT-5 nano) showed significantly reduced redirection quality and benign utility, despite maintaining low harmful compliance.
Statistical Significance: After multiplicity adjustment (Holm correction), significant differences were found between certain pairs (e.g., Claude Haiku 4.5 vs. GPT-5 nano), but no tested contrast between the Claude/Gemini snapshots and GPT-5 or GPT-5.5 survived correction.
Coverage Issues: Two snapshots (Claude Opus 4.6 and 4.7) lacked complete domain coverage in the judging layer, rendering their BGS non-estimable. This was attributed to judging layer failures, not target model behavior.
Formal Verification: Two PRISM abstractions verified the authorization separation and lifecycle invariants, confirming that under the specified structural assumptions, unprivileged components cannot reach authorized disclosure states.

Significance and Claims
The paper explicitly states that its principal contribution is a precautionary control architecture, not a definitive leaderboard or a universal ranking of AI safety.

Safety-First Architecture: SAGE demonstrates that catastrophic-risk limits can be made primary constraints via a lexicographic policy, ensuring that utility is only optimized among safety-feasible actions.
Defense-in-Depth Limits: The formal results highlight that adding layers of defense is insufficient without measured diversity; common-cause failures can render multiple layers ineffective.
Authorization Separation: The framework successfully separates conservative response control from exceptional organizational action, ensuring that safety filters do not automatically infer malicious intent or trigger external reporting.
Modest Empirical Claims: The authors emphasize that the observed low harmful-compliance range (0.0000–0.0333) is a conservative, protocol-bound view from a single generation per prompt without tools, history, or adaptive jailbreaks. They explicitly state this is not an upper bound on operational assistance.
No Deployment Certification: The paper concludes that no benchmark certifies deployment. The observed results are endpoint-specific and snapshot-specific. A model satisfying these thresholds must still undergo broader adversarial testing, multi-turn auditing, and human-uplift studies before deployment.

The paper serves as a blueprint for a "defense-in-depth" lifecycle where safety is enforced through signed manifests, formal invariants, and strict authorization cuts, rather than relying solely on detector accuracy.