💻 computer science

Stop Means Stop: Measuring and Repairing the Enforcement Gap in Agent-Framework Control Primitives

This paper reveals a critical enforcement gap in six major LLM-agent frameworks where control primitives like approval gates and timeouts fail to prevent side effects during pauses or cancellations, and proposes SOUNDGATE, a mechanically verified, high-performance Rust-based gate that guarantees complete mediation of all side effects across diverse frameworks.

Original authors: Sajjad Khan

Published 2026-07-20

📖 7 min read🧠 Deep dive

CC BY 4.0

Original authors: Sajjad Khan

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine a world where your computer doesn't just follow orders but starts making its own plans. This is the realm of AI Agents: smart programs that can read emails, book flights, or even move money around. But because these agents are powerful, we need a way to stop them if they get confused or start doing something dangerous. This is where Human-in-the-Loop (HITL) comes in. Think of it as a "pause button" that forces the AI to ask a human, "Hey, I'm about to send this email. Is that okay?" The human says "Yes" or "No," and the AI waits for that answer before doing anything irreversible.

For this safety system to work, everyone assumes a simple rule: Stop means Stop. If the AI hits the pause button, nothing happens until the human replies. It's like a traffic light turning red; all cars must stop, and no car should sneak through the intersection while the light is red. If the light is red, the intersection is empty. This paper investigates whether the software frameworks (the "traffic controllers" for these AI agents) actually keep this promise. The researchers wanted to know: When the AI pauses to ask for permission, is the rest of the system truly frozen, or is it secretly letting other things happen in the background?

The Great "Sibling" Leak

The researchers, led by Sajjad Khan, discovered that the "Stop means Stop" promise is broken in almost every major AI framework they tested. They found a sneaky bug they call the "Sibling Leak."

Imagine an AI agent is a busy kitchen. The chef (the AI) is preparing two dishes at the same time: Dish A is a spicy curry that needs the owner's permission before serving, and Dish B is a simple salad. The chef stops to ask the owner about the curry. In a perfect kitchen, the whole kitchen freezes until the owner says "Go." But in the kitchens these researchers tested, the kitchen didn't freeze. While the chef was waiting for the owner's answer about the curry, the salad (Dish B) was still being chopped, plated, and served to the customer.

Even worse, if the owner looked at the curry, said "No, don't serve that," and the chef tried to stop, it was too late. The salad had already been served. The "stop" command only froze the specific branch of the chef's brain thinking about the curry, but the other branches kept running wild.

The team tested six different frameworks (the software toolkits developers use to build these agents) and found this leak in five of them. It happened across different types of computer systems and programming languages. It wasn't just one bad code snippet; it was a pattern that kept happening over and over.

The Other Glitches

The "Sibling Leak" wasn't the only problem. The researchers found three other ways the safety brakes failed:

The Replay Double-Count: If the system paused and then started again, sometimes it would accidentally do the same task twice, like charging a credit card two times because it thought the first time didn't count.
The Cancellation Orphan: If a human told the AI to "Stop!" while it was in the middle of a long task, the AI would say "Okay, I'm stopping," but the task would finish anyway in the background, like a runner who hears the whistle but keeps running to the finish line.
The Timeout Zombie: If a task took too long and the system said "Time's up!", the task would still finish its job after the deadline passed, like a zombie that keeps walking even after the sun has risen.

Does This Actually Happen?

You might think, "Well, maybe AI just doesn't do two things at once very often." The researchers checked this, too. They found that while AI agents usually do things one by one in normal situations, they do sometimes try to do multiple things at once, especially when the task is complex or when someone tries to trick the AI.

In their tests, they found that frontier AI models (the smartest ones available) would accidentally create these "sibling" situations about 14% of the time on certain tasks. However, the critical finding is about what happens when they do try to do two things at once. In a live test with real AI models and real software, whenever the AI generated a plan that tried to do two things simultaneously while waiting for a human, the system failed 100% of the time. Out of 1,200 total runs, 215 resulted in a "leak" where an action happened even though the system was paused. The failure wasn't that the AI always tried to do two things; it was that when it did, the safety gate was completely useless.

The Fix: The Bouncer at the Door

So, how do you fix a kitchen where the chef keeps sneaking food out while waiting for permission? You can't just tell the chef to "be better," because the kitchen itself is built to let things happen in parallel. Instead, the researchers built a new kind of safety guard called SOUNDGATE.

Think of SOUNDGATE as a bouncer standing at the front door of the kitchen.

The Old Way: The chef (the AI) would just walk out the door with the food. If the owner said "Stop," the chef might be too far away to hear it.
The SOUNDGATE Way: Every single dish (every action) must stop at the bouncer before it leaves the kitchen. The bouncer holds a list of what is allowed.
- If the owner says "Wait," the bouncer holds the food.
- If the owner says "No," the bouncer throws the food away.
- If the owner says "Yes," the bouncer lets it out.
- If the chef tries to sneak out a second time (a replay), the bouncer checks the list and says, "You already did this," and stops it.
- If the chef tries to leave after the "Stop" order, the bouncer blocks the door.

The researchers built this bouncer using a very strict, mathematically verified language called Rust. They proved with computer logic that the bouncer cannot make a mistake. They tested it on all six frameworks, and it worked perfectly. When they used SOUNDGATE, zero leaks happened. The bouncer held the line every single time.

Why This Matters

The paper doesn't claim that AI is dangerous or that we should stop using it. Instead, it shows that the safety tools we are currently using have a hidden hole. It's like having a seatbelt that looks great but doesn't actually lock when you crash.

The researchers found that while the hole exists, it's often "latent"—meaning it's there, but we don't always see it because AI usually does things slowly and one by one. However, as AI gets faster and starts doing more complex, multi-step tasks, or if someone tries to trick it, that hole becomes a giant gap.

The good news is that the fix is simple and doesn't require rebuilding the whole kitchen. You just add the bouncer (SOUNDGATE) at the door. It's a small addition that makes the whole system safe again, ensuring that when a human says "Stop," the machine actually stops.

Technical Summary: Stop Means Stop: Measuring and Repairing the Enforcement Gap in Agent-Framework Control Primitives

1. Problem Statement

Large Language Model (LLM) agents increasingly perform irreversible actions on external systems (e.g., sending emails, authorizing payments, deploying code). To mitigate risk, major agent frameworks ship control primitives: Human-in-the-Loop (HITL) approval gates, run cancellation, and execution timeouts.

Practitioner guidance and vendor documentation imply a barrier contract: when a run is paused for approval, cancelled, or timed out, no gated side effect executes until a decision is made. If an operator rejects a request, the action should not have occurred.

The paper identifies a critical enforcement gap: this barrier contract does not hold in practice. The authors demonstrate that in widely used frameworks, the "stop" semantics are violated due to concurrency and execution model design. Specifically:

Sibling Leak: When an approval gate pauses one branch of a parallel execution step, sibling branches continue executing. If a sibling performs a side effect during the pause, a subsequent human rejection is powerless to prevent it.
Replay Double-Execution: Upon resuming a paused run, some frameworks re-execute steps from the beginning, causing effects performed before the pause to execute a second time.
Cancellation Orphans: When a run is cancelled, asynchronous worker threads may continue to completion, causing effects to land after the cancellation signal is observed.
Timeout Zombies: Effects may complete and land after a timeout has been reported to the caller.

These failures are not isolated bugs but recurring patterns across four distinct execution models (Pregel/BSP, event buses, message-passing fan-out, and parallel tool calls) and two language runtimes (Python and Node.js).

2. Methodology

Measurement (Model-Free Probes)

The authors developed model-free differential probes to isolate framework behavior from LLM nondeterminism.

Mechanism: Minimal workflows were constructed where the control primitive (pause, cancel, timeout) was the only variable. Side effects were instrumented as appends to an in-process event log.
Scope: Six open-source frameworks were evaluated (LangGraph, LlamaIndex Workflows, Microsoft Agent Framework, OpenAI Agents SDK, CrewAI, and LangGraph.js).
Verification: Probes were executed across multiple independent environments and framework versions to ensure reproducibility. A randomized structural sweep of 1,000 workflows on LangGraph confirmed the leak rate is deterministic based on execution topology.

Exposure and Corroboration

Model Emission: The study measured how often frontier models (GPT-4o, Claude, Gemini, DeepSeek, Llama) emit the specific "leak-triggering" plan shape (a consequential tool call sharing a turn with benign siblings). Rates varied by model and task, with GPT-4o emitting the shape in up to 14% of runs.
End-to-End Execution: Live models were driven through real frameworks to measure the unconditional probability of a leak ( $P(\text{leak})$ ). Results showed that once the plan shape is emitted, the framework leaks the effect with probability 1.00 ( $P(\text{leak} | \text{emitted}) = 1.00$ ).
Public Corpus: A corpus of 13 public incidents from issue trackers was compiled, providing qualitative corroboration that these failures occur in the wild.

Repair and Verification

The authors proposed SOUNDGATE, an environment-external effect gate.

Design: A small, language-independent arbiter (implemented in Rust) that sits outside the framework's control flow. Every side effect must be submitted to SOUNDGATE for admission before execution.
Contract: The repair relies on a complete-mediation contract: every side-effecting path must submit to the gate. This is enforced via wrapper discipline, static linting, and structural kernel-level routes (network namespaces and eBPF hooks) that refuse unmediated network egress.
Formal Verification: The admission core was mechanically verified using:
- Verus: Proving safety properties over a sequential model.
- TLA+/TLC: Exhaustive model checking of concurrent interleavings (up to $7.5 \times 10^7$ states).
- TLAPS: Inductive proofs for unbounded runs.
- Loom: Bounded model checking of the deployed concurrent Rust code.
- Differential Conformance: $1.2 \times 10^7$ operations comparing the deployed code against the formal model with zero divergences.

3. Key Contributions

Measurement (C1): A characterization of a cross-framework enforcement gap in shipped control primitives. The paper isolates a "sibling leak" and confirms replay, cancellation, and timeout violations across five frameworks (four independent designs plus a cross-language port) and two runtimes.
Qualitative Corroboration (C2): Evidence that the hazard is reachable in practice. Real models emit the triggering plan shape at non-negligible rates, and live end-to-end measurements confirm that $P(\text{leak} | \text{emitted}) = 1.00$ . A public corpus of 13 incidents supports the existence of these failures in production.
External Repair Mechanism (C3): SOUNDGATE, an environment-external gate whose admission core is model-verified and differentially tested. It enforces four properties:
- Hold-until-decided: Prevents sibling leaks.
- Reject-cancels: Ensures rejected effects never execute.
- Dedup-on-replay: Prevents double-execution on resume.
- Fence-on-cancel: Prevents zombie effects after cancellation.
  The repair was demonstrated end-to-end on all six frameworks, covering all four execution models and both runtimes.
Evaluation (C4): Demonstration that the gate suppresses violations under benign and adversarial input (including prompt injection) while maintaining low overhead (~~1 ms per write) and high throughput (~~12k admissions/sec in durable mode).

4. Results

Violation Rates: In the evaluated frameworks, the sibling leak and reject-after-effect violations reproduced in every framework capable of expressing the configuration.
Live Leak Rates: In end-to-end experiments with live models:
- GPT-4o: 44% of runs leaked an effect during the pause without the gate.
- Claude: 9% of runs leaked.
- Llama 3.3: 19% of runs leaked.
- DeepSeek: 0% (due to sequential disposition).
- Mediated Rate: With SOUNDGATE, the leak rate dropped to 0% across all models and runs.
Performance:
- Latency: Median admission cost was ~1.1 ms per write (median episode latency was statistically unchanged).
- Throughput: A single node sustained ~12,000 admissions per second in durable (WAL) mode with group commit.
- Correctness: On the $\tau$ -bench retail benchmark, the gated agent completed episodes with zero false refusals of legitimate writes.

5. Significance and Claims

The paper claims that the "stop means stop" assumption, which practitioners rely on for safety, is a contract mismatch rather than a formal guarantee in current agent frameworks. The significance lies in three areas:

Empirical Characterization: The paper provides the first systematic, cross-framework measurement of these control-plane failures, proving they are not isolated implementation bugs but structural issues arising from the tension between parallelism and barrier semantics.
Feasibility of External Enforcement: It demonstrates that a robust barrier can be enforced outside the framework's control flow. By placing the enforcement point at the tool boundary (via an external gate), the system can guarantee safety properties regardless of the underlying framework's scheduler or concurrency model.
Verification and Adoption: The repair (SOUNDGATE) is not just a theoretical proposal; it is a verified, deployable component. The admission core is mechanically verified, and the integration cost is minimal (a ~20-line wrapper per framework).

Modesty of Claims:

The paper explicitly states it does not claim to fix prompt injection or defend against compromised tools; it assumes the "complete-mediation contract" (that all effects are routed through the gate).
It acknowledges that on naturalistic tasks, current frontier models often serialize writes, making the gap "latent" rather than prevalent. However, it argues that the barrier is necessary for scenarios involving adversarial input, multi-effect tasks, or explicit parallel configurations.
The repair is conditional on the complete-mediation contract; if a tool bypasses the gate (e.g., via an unwrapped path), the guarantee does not hold. The paper addresses this via structural kernel routes (eBPF/namespaces) that force network egress through the gate, converting potential silent leaks into loud availability errors.

In conclusion, the paper argues that for agent systems with irreversible tool access, environment-external effect gating is a robust default for ensuring that human decisions (approval, rejection, cancellation) are mechanically respected, closing a critical safety gap that current frameworks fail to address.