Governed Reasoning for Institutional AI

1. Problem Statement

The paper identifies a critical gap between general-purpose AI agents and institutional decision-making. While current agent architectures (e.g., ReAct, Plan-and-Solve) excel at decomposing goals and adaptive sequencing, they lack the structural properties required for high-stakes institutional contexts (e.g., regulatory compliance, clinical triage, loan modifications).

Key Limitations of Current Agent Architectures:

• Conversational Authority: Authority boundaries are inferred from conversation rather than structurally enforced, leading to "sycophantic capitulation" (yielding to incorrect challenges) and suppression of minority correct opinions.
• Monolithic Reasoning: Reasoning is often a single loop, making it difficult to inspect individual steps or audit specific epistemic operations.
• Post-Hoc Accountability: Audit trails are reconstructed from prompts and outputs rather than being generated endogenously during computation.
• Silent Errors: Systems often produce confident but incorrect determinations without triggering human review, a failure mode documented in multi-agent systems (e.g., Gu et al. [18]).

The Core Thesis: Institutional decisions require a distinct architecture where governance, inspectability, and persistent accountability are structural properties of the execution model, not just monitoring layers applied to outputs.

2. Methodology: The Cognitive Core Architecture

The paper proposes Cognitive Core (CC), a decision substrate designed around four theoretical commitments derived from institutional theory and formal methods.

A. Theoretical Commitments

1. Distinct Problem Class: Institutional decisions require consequential durability, formal explanation obligations, and role-bounded authority.
2. Governance on Structure: Governance must attach to the typed output of each reasoning step, not just the final result.
3. Coordinated Specialization: Institutional work is adaptive; the system must reason backward from goals to evidence and adapt the investigation path dynamically.
4. Endogenous Accountability: The audit trail must be produced by the computation (as a primary output) rather than derived from logs afterward.

B. Core Components

1. Cognitive Primitives (The Vocabulary):
   A set of nine typed, atomic epistemic operations, each with defined input/output contracts and governance profiles:

   • Retrieve, Classify, Investigate, Verify, Challenge, Reflect, Deliberate, Govern, Generate.
   • The Reflect Primitive: A unique metacognitive operation that reasons about the reasoning process itself. It acts as a "post-challenge guard," distinguishing between genuine epistemic vulnerabilities (requiring revision) and adversarial pressure/sycophancy (requiring preservation of the original determination).

2. Governance Model (Four-Tier System):
   A structural enforcement mechanism where review requirements are conditions of execution. The govern primitive reads the accumulated epistemic state and assigns a tier:

   • AUTO: High confidence, no flags (completes immediately).
   • SPOT CHECK: Standard quality monitoring (10% sampling).
   • GATE: Degraded state or challenge cycles (suspends for mandatory human review).
   • HOLD: Insufficient state or blocking flags (suspends with SLA enforcement).
   • Key Feature: Tiers lock for the instance lifetime; escalation is strictly upward.

3. Epistemic State Architecture:
   Replaces the single LLM-reported confidence scalar with a three-layer framework:

   • Layer 1 (Mechanical): Deterministic signals (e.g., evidence completeness, rule coverage) that cannot be inflated by the LLM.
   • Layer 2 (Judgment): LLM-reported reasoning quality and outcome certainty (weighted 40%).
   • Layer 3 (Coherence): Cross-step flags detecting contradictions or confidence drops.
   • Result: A "warranted" flag is computed based on these signals, not just the LLM's self-assessment.

4. Execution Modes:

   • Workflow Mode: Declared sequence in YAML (fixed path).
   • Agentic Mode: Orchestrator selects primitives dynamically based on goals and evidence, but is constrained by hard substrate-level rules (e.g., must_include, max_steps) and the governance pipeline.

5. Audit Model:
   A SHA-256 hash-chained ledger where every primitive output, orchestrator decision, and governance action is recorded immutably. The reasoning trace is endogenous to the computation.

3. Key Contributions

1. Architectural Substrate: A design that enables autonomous epistemic trajectories within structural guardrails, separating the "what" (reasoning) from the "how" (governance).
2. Metacognitive Guard: The reflect primitive specifically addresses the failure mode of "sycophantic capitulation" by requiring a reasoned argument for revision rather than mere challenge pressure.
3. Governability Metric: Introduces governability as a primary evaluation axis: how reliably a system knows when it is not right (i.e., routing errors to human review) vs. just accuracy.
4. Configuration Economics: Demonstrates that new domains can be configured via YAML (workflow/domain) and JSON (case data) without engineering code changes, requiring only domain expertise.
5. Reference Implementation: A 15,000-line Python implementation (Cognitive Core v0.1.0) using DEVS formalism, supporting multi-provider LLMs and a full governance pipeline.

4. Results: Comparative Benchmark

The paper presents a benchmark comparing Cognitive Core (CC) against ReAct and Plan-and-Solve (neutral-framed baselines) on an 11-case balanced prior authorization appeal dataset.

Key Findings:

• Accuracy: CC significantly outperformed baselines, particularly on UPHOLD and PARTIAL cases where baselines suffered from "approval prior bias" (defaulting to overturning denials even when criteria were met).
• Governability: This is the most significant result. CC produced zero silent errors. Its single unique reasoning error (Case G003) and the shared hard case (B001) were both routed to GATE for human review.
• Baseline Failure: Both ReAct and Plan-and-Solve produced 5–6 incorrect determinations that executed without any review signal. Even when Plan-and-Solve correctly identified criteria non-compliance in its planning phase, it failed in the execution phase due to disposition commitment bias.
• Tier Calibration: CC successfully routed 45% of cases to SPOT CHECK (no mandatory review), demonstrating that governance does not necessarily eliminate efficiency gains.

5. Significance and Implications

• Shift from Accuracy to Trustworthiness: The paper argues that for institutional AI, governability (knowing when to stop and ask for help) is more critical than raw accuracy. An accurate system that executes silently on errors is less trustworthy than a system that flags its own uncertainty.
• Structural vs. Conversational Authority: It proves that institutional trust cannot be achieved through better prompting alone; it requires structural enforcement of authority boundaries and reasoning steps.
• Metacognition as a Safety Mechanism: The reflect primitive demonstrates that AI systems can be architected to resist social pressure and adversarial framing, preserving correct minority opinions against confident but wrong challenges.
• Institutional Intelligence: The work suggests that institutions do not need AI that mimics individual human intelligence, but rather systems that embody institutional intelligence—processes that are bounded, auditable, and accountable by design.

In conclusion, Cognitive Core offers a viable architectural path for deploying AI in high-stakes environments, transforming AI from a "black box" generator of outputs into a governed, inspectable, and accountable reasoning engine.