Governing Actions, Not Agents: Institutional Attestation as a Governance Model for Autonomous AI Systems

Technical Summary: Governing Actions, Not Agents

Problem Statement
Autonomous AI agents increasingly perform consequential, irreversible actions (e.g., clinical prescribing, software deployment) without per-step human approval. Existing governance approaches typically instrument the agent's runtime to intercept tool calls, classify behavior, and enforce policy based on observed execution context (tool names, parameters, response shapes). However, these mechanisms fail to govern actions whose correctness depends on facts about the state of the world residing in authoritative external systems (e.g., whether a drug interaction was checked, a build passed, or a license is valid). The paper argues that monitoring an agent's reasoning or tool invocation mechanics is insufficient for high-stakes actions where the validity of the action relies on external, independent verification.

Methodology: Institutional Attestation Model
The paper proposes a computational governance model derived from the institutional governance of human autonomous actors (e.g., physicians, judges). Instead of governing the agent's deliberation, the model governs the action at the point of execution by requiring independently attested evidence. The model is implemented as a "Zero-Trust Action Hub" (proof-of-concept available at github.com/jsalfeld/zta-hub) and operates via the "Courier Pattern":

1. Intent Declaration: The agent requests authorization for a specific action. The Governance Hub issues a unique, cryptographically random intent identifier (binding token) and returns a list of required attestations defined in machine-readable skill contracts.
2. Evidence Collection: The agent acts as a courier, contacting independent authoritative sources (oracles) to collect signed attestations. Each oracle verifies a specific precondition and returns a signed envelope (e.g., using Ed25519) containing the source ID, intent ID, expiry, and payload. The agent cannot fabricate these signatures as it does not hold the oracles' private keys.
3. Submission and Evaluation: The agent submits the collected attestations to the Hub. The Hub performs a verification pipeline:
   • Signature Verification: Validates signatures against pre-registered public keys.
   • Intent Binding: Confirms the intent ID in each attestation matches the declared request, preventing replay or substitution.
   • Freshness: Checks that attestations are within their validity window.
   • Completeness: Ensures all required attestations are present.
   • Context Assembly: Merges verified payloads into a policy context.
4. Deterministic Policy Evaluation: The Hub evaluates a declarative policy (e.g., using Cedar or Rego) against the assembled context. The default posture is "deny."
5. Conditional Authorization: If the policy permits, the Hub either executes the action or issues a signed, narrowly-scoped capability token.
6. Audit: The decision, all supporting evidence, and the applied rule are appended to a tamper-evident audit log (using hash-chain or Merkle-tree techniques), enabling independent re-verification by third parties.

Key Contributions

• Formalization of Institutional Governance: The paper formalizes the pattern of governing actions rather than agents, translating human institutional precedents (separation of duty, independent attestation) into a computational architecture for AI.
• The Courier Pattern: A mechanism where the agent retains full autonomy over planning and reasoning but holds no execution authority for governed actions, acting solely to assemble evidence from independent sources.
• Intent Binding and Transaction Integrity: A cryptographic framework where attestations are bound to a specific action intent, preventing replay attacks and ensuring evidence cannot be detached from its context.
• Action Composition: The model allows governance to compose; a successful action yields a signed execution receipt that can serve as a precondition (verified evidence) for subsequent actions.
• Scalable Governance Discovery: Governance requirements are published as machine-readable skill contracts, allowing new actions, attesters, and oracles to be introduced without modifying the agent or the Hub's evaluation logic.

Results and Illustrations
The paper presents a proof-of-concept implementation and illustrates the model with two examples:

1. Software Deployment: An agent seeking to deploy code must collect signed attestations from a CI pipeline (tests passed), a code review oracle (approvals), and a security scan oracle (vulnerability checks). The Hub verifies these against a Cedar policy before authorizing the deployment.
2. Clinical Prescribing: An agent prescribing medication must collect attestations from an EHR (patient status), a drug interaction oracle, and a DEA license oracle. The policy includes logic for action composition, requiring evidence that a prior action (e.g., releasing medical records for transfer patients) was successfully executed under the same governance regime.

Significance and Claims
The paper claims to articulate a composition of well-established primitives (zero-trust verification, capability security, deterministic policy evaluation, tamper-evident logs) into a unified institutional governance model. Its significance lies in addressing a specific failure mode of current safety layers: actions that are locally well-formed (correct tool, plausible parameters) but substantively incorrect because they rely on unverified external facts.

The model is particularly relevant to the "rogue agent" problem, where a compromised agent operates within authorized scopes but pursues adversarial objectives; runtime behavioral monitoring is structurally ineffective here because individual actions conform to expected patterns. By requiring evidence-gating based on independently attested facts, the model ensures that consequential actions are only permitted when real-world preconditions are verified.

The authors explicitly state that the open problem is not primarily technical but organizational: determining which actions warrant this level of governance and what evidence is required. The model guarantees the faithful enforcement of a stated policy but does not assure the correctness of the policy itself or the alignment of the agent's declared intent with human goals. It is designed to complement, not replace, runtime governance mechanisms, addressing the "boundary" of action rather than the "path" of reasoning.