💻 computer science

Runtime Continuation after Faults in Partially Executed Agent Workflows

This paper proposes a runtime continuation mechanism for partially executed language-model agent workflows that ensures safe recovery from faults by reconstructing a trusted frontier, deriving residual obligations from a canonical contract, and advancing state only through authorized effects supported by valid, fresh, and uniquely matching evidence.

Original authors: Li Zeng, Chuanwu Yang, Qiang Zhao, Qinde Chen, Deyang Qu

Published 2026-09-22

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Li Zeng, Chuanwu Yang, Qiang Zhao, Qinde Chen, Deyang Qu

Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

In the modern digital landscape, artificial intelligence has evolved from a system that simply writes text to one that actively performs tasks. These intelligent agents can search the web, edit files, send messages, and update databases on a user's behalf. They operate by following a sequence of steps, often called a workflow, where each action changes the state of the real world. However, this ability to act creates a unique vulnerability. If an agent makes a mistake, gets confused by a trick, or encounters a system error after it has already changed a file or sent a message, simply trying again or starting over can be dangerous. Replaying a failed sequence might repeat a harmful action, while continuing from a confused state might build on a false assumption. The core challenge is determining exactly which part of the history is trustworthy, what work remains unfinished, and what proof is needed to safely move forward without repeating errors or falling for deception.

Researchers Li Zeng and his team at the Institute of Information Technology of the National Immigration Administration and the Hong Kong University of Science and Technology have developed a new method to solve this problem. They call their system AgentMirror. Instead of treating a broken workflow as a simple error to be fixed by a new guess, AgentMirror treats the recovery process as a strict, step-by-step verification of reality. The system operates on the principle that once an agent has performed an action, the history of that action becomes a fixed point of truth. If the agent crashes or is tricked after that point, the system does not ask the artificial intelligence to decide what happened. Instead, it looks at a verified record of what actually occurred, identifies exactly which tasks are still left to do, and demands concrete proof before allowing any new action to change the state of the world again.

The researchers built this system around a concept they call a "trusted frontier." Imagine a line drawn in the sand that marks the exact moment where the agent's history is confirmed to be safe and correct. Everything before this line is accepted as true; everything after is unverified. When a fault occurs, the system does not let the artificial intelligence rewrite this line or pretend that a mistake didn't happen. It reconstructs the state of the agent based only on the verified history up to that line. From this point, the system calculates a list of "residual obligations"—the specific, mandatory tasks that the agent still needs to complete to finish the job. It then presents this list to the artificial intelligence as a guide, telling it what work is left, but without giving the intelligence the power to just do it.

The critical innovation lies in how the system handles the next step. When the artificial intelligence proposes a new action, that proposal is treated as untrusted. The system forces the action through a standard security check, known as "ordinary admission," to ensure it is allowed to run. If the action executes, the system does not immediately accept that the task is done. It waits for a "runtime receipt," which is a digital certificate issued by a trusted authority confirming that the action actually happened and was observed correctly. This receipt must be fresh, meaning it was just created, and it must be bound to the current trusted frontier, meaning it cannot be a replay of an old action or a fake from a different session. Only when this receipt is verified and matches exactly one of the remaining tasks does the system allow the official record of the agent's progress to move forward.

To test their idea, the researchers used a benchmark called AgentDojo, which simulates various tasks and deliberately introduces faults and attacks to see how agents behave. They compared their AgentMirror system against other recovery methods, such as simply retrying the last step or letting the agent guess its way out of trouble. The results showed that AgentMirror was significantly better at completing tasks safely. It successfully recovered from faults without allowing unauthorized actions to slip through, and it prevented the agent from falling for trick instructions that tried to make it repeat harmful operations. The study demonstrated that by separating the guidance given to the artificial intelligence from the authority to execute actions, and by requiring fresh evidence for every step, the system could maintain a secure path forward even when the agent itself was compromised.

The researchers also explored what would happen if they removed specific parts of their safety rules. They found that if they skipped the step of verifying the receipt, the system would accept fake or old evidence, leading to incorrect progress. If they allowed the recovery guide to act as a permission slip, the system would let untrusted actions run. If they did not check that the action matched exactly one specific task, the system might close the wrong task or leave work unfinished. These tests confirmed that every part of their process is necessary; no single check can replace the others. The system works because it forces a chain of trust where the artificial intelligence can suggest, but the system decides, and only verified reality can change the record.

This work does not claim to make artificial intelligence perfect or to solve every possible failure. The system relies on the initial contract or set of rules being correct; if the rules themselves are wrong, the system will faithfully follow the wrong path. It also cannot undo actions that have already been performed in the real world if the proof of those actions is lost. However, it provides a robust framework for managing the moment when things go wrong. By treating the recovery process as a strict transition from a known, safe state to a new, verified state, AgentMirror offers a way to keep intelligent agents working safely even when they encounter errors or malicious interference. The study concludes that the key to safe continuation is not better guessing, but better verification.

Technical Summary: Runtime Continuation after Faults in Partially Executed Agent Workflows

1. Problem Statement

Large Language Model (LLM) agents increasingly interact with the world by modifying durable external state through tools (e.g., editing files, updating databases, sending messages). When a fault occurs after such an effect has been realized—whether due to a model error, a tool failure, or a malicious prompt injection—standard recovery mechanisms face a critical semantic gap.

Existing approaches typically address parts of the lifecycle: runtime authorization decides if a proposed effect can execute; diagnosis and replanning suggest what to try next; and provenance systems record what happened. However, none of these mechanisms alone defines how a partially executed run should re-enter its canonical task state safely. Specifically, after a fault, it is unclear:

Which execution prefix remains trustworthy?
Which mandatory obligations are still open?
What evidence is required to justify further progress without repeating irreversible operations or accepting invalid assumptions?

Replaying a trajectory risks repeating irreversible side effects, while continuing from the latest model message risks carrying forward invalid assumptions. The paper identifies the missing semantic unit as the post-fault transition from a trusted runtime state to the next admissible task state.

2. Methodology: AgentMirror

The authors instantiate a runtime mechanism called AgentMirror to solve the post-fault continuation problem. The core philosophy is to treat recovery not as another model proposal, but as a constrained state transition over authenticated execution history.

2.1 Core Concepts

Trusted Frontier ( $F$ ): A runtime-authenticated commitment to an executed prefix. It is reconstructed from a ledger of committed events, not selected by the model. It represents the "safe" point up to which the system trusts the history.
Residual Obligations: Derived from a canonical task contract ( $K$ ), these are the mandatory obligations remaining after the trusted frontier. The contract defines dependencies, targets, and expected evidence.
Recovery Context ( $C_F$ ): A projection of the trusted state (frontier, open obligations, invalidated events, budget) provided to the black-box model. Crucially, this context is non-authorizing; it guides proposal generation but grants no permission to execute.
Runtime Receipts ( $\rho$ ): Authoritative evidence issued after execution. A receipt is only valid if it is fresh, bound to the current frontier/run, and uniquely matches an open obligation.

2.2 The Transition Lifecycle

The mechanism enforces a strict separation of concerns across four stages (illustrated in Figure 1 of the paper):

Proposal: The external integration presents a normalized candidate action ( $a$ ) based on the recovery context.
Ordinary Admission: The candidate must pass standard authorization checks ($Auth$) before execution. Recovery context does not bypass this.
Execution & Evidence: The tool executes the effect. If successful, a trusted issuer produces a receipt ( $\rho$ ).
Canonical State Update: The transition component updates the canonical state ( $S$ $S$ ) only if:
- The receipt is valid, fresh, and bound to the current frontier.
- The receipt uniquely matches exactly one open obligation (Singleton Check).
- All predecessors of that obligation are satisfied.

If any condition fails (e.g., the receipt is stale, matches multiple obligations, or is from a different frontier), the system takes the identity branch: the canonical state remains unchanged. This prevents "invalid progression" even if an external effect occurred; the system simply does not recognize it as a valid step in the canonical task.

2.3 Formal Guarantees

The paper formalizes the lifecycle as a transition relation. It proves that under the assumption of a trusted contract and trusted runtime components (ledger, issuer, admission), the system guarantees:

Admissible Progression: State advances only if the effect was authorized, realized, and supported by fresh, bound evidence.
Non-Substitutability: The conditions (admission, execution evidence, receipt validity, frontier binding, singleton matching) are distinct. Removing any single condition admits a specific class of invalid progression (e.g., removing frontier binding allows cross-prefix evidence; removing singleton matching allows ambiguous closure).

3. Key Contributions

The paper makes three primary contributions:

Post-Fault State-Transition Semantics: A formal model centered on a runtime-established frontier, residual obligations, non-authorizing recovery context, and evidence-bound progression.
Typed Separation of Concerns: A clear distinction between guidance (recovery context), authority (admission), execution evidence (receipts), and task correctness. The paper provides "condition-removal witnesses" demonstrating why these conditions cannot substitute for one another.
Runtime Mapping and Evaluation Scope: A framework for evaluating black-box agent workflows that separates task utility, attack success, secure utility, and invalid transitions. It defines specific metrics for RQ1 (Safety/Utility), RQ2 (Generality), and RQ3 (Condition Attribution).

4. Evaluation Scope and Results

The paper outlines an evaluation strategy using the AgentDojo benchmark, which provides stateful tool environments and external task oracles.

RQ1 (Overall Safe Recovery Effectiveness): Measures whether AgentMirror recovers tasks without unauthorized effects and improves completion rates compared to baselines like "No Recovery" or "Basic Retry." Metrics include Utility, Attack Success Rate (ASR), and Secure Utility.
RQ2 (Generality): Tests effectiveness across different AgentDojo applications, fault families, and black-box models. It evaluates whether the mechanism remains effective regardless of the specific model or provider adapter used below the normalized proposal boundary.
RQ3 (Condition Attribution): Uses ablation studies to remove specific transition or ownership conditions (e.g., removing ordinary admission, weakening receipt freshness). The goal is to show that each removed condition admits a distinct class of invalid progression, validating the necessity of the full set of constraints.

Note: The paper presents the methodology and evaluation design but does not report specific numerical results (e.g., "95% success rate") in the provided text. The focus is on the semantic correctness and the theoretical guarantees of the transition model.

5. Significance and Claims

The paper claims to address a "missing transition" in agent safety literature. While prior work covers pre-fault controls, in-fault monitoring, or post-effect records, AgentMirror specifies the contract-relative transition that connects them after partial execution.

Contract-Relative Guarantee: The system guarantees internal consistency relative to the task contract. It does not claim global task correctness if the contract itself is flawed (e.g., omits an obligation), but it ensures that the runtime state advances only when the contract's evidence requirements are met.
Defense Against Authority Confusion: A key contribution is preventing "recovery metadata" from being mistaken for "execution authority." The recovery context guides the model but cannot mint credentials or bypass admission.
Modesty on Limitations: The authors are explicit about limitations:
- The guarantee is relative to a trusted contract; a bad contract leads to a consistent but wrong task state.
- There is no unconditional liveness guarantee; if no candidate passes admission or matches an obligation, the system halts progress.
- It does not handle irreversible external effects that lack idempotency or compensation mechanisms outside the canonical transition.
- Provider parsing and adapter correctness remain external concerns.

In summary, AgentMirror provides a rigorous semantic framework for resuming black-box agent workflows after faults, ensuring that state progression is driven by authenticated evidence and strict contract adherence rather than model confidence or unverified recovery plans.