Technical Summary: Governance from Metrics and the govllm Framework

1. Problem Statement

Current approaches to AI compliance treat conformity as a binary, audit-time verdict rather than a continuous, measurable property of production systems. This "compliance fiction" is structurally ill-suited to the requirements of the EU AI Act, which mandates ongoing human oversight and the detection of emergent behavioral drift in deployed systems. Static assessments (e.g., documentation written once, periodic audits) fail to capture the dynamic nature of agentic and generative AI, where behavior evolves with context, usage patterns, and model updates. Furthermore, existing governance frameworks often lack operational tooling to monitor compliance at runtime, while existing evaluation frameworks are optimized for latency and cost rather than regulatory signal generation.

A critical structural tension exists: in regulated environments with data sovereignty requirements (e.g., GDPR Article 44), routing compliance evaluation through external APIs may itself violate the obligations being assessed. Consequently, on-premise evaluation is a regulatory necessity, not merely a design preference.

2. Methodology: Governance from Metrics

The paper introduces Governance from Metrics, a principle asserting that regulatory compliance must be treated as a continuous signal derived from production observability, rather than a static audit verdict. To operationalize this, the authors present govllm, an open-source, on-premise runtime framework.

Core Architectural Components

• Governance Profiles: An ordered set of compliance criteria (e.g., transparency, data privacy, non-manipulation) associated with weights and minimum thresholds. These profiles map legal obligations to measurable technical metrics.
• Profile-as-Jury: Instead of a single monolithic judge, the system employs a panel of specialized LLM judges. Each regulatory criterion is assigned to a specific LLM judge (e.g., a data protection officer role mapped to a specific model). This design enforces generator/evaluator separation to prevent self-preference bias.
• Runtime Evaluation Pipeline:
  • Chat Module: Routes user prompts to active models and triggers asynchronous judge evaluation.
  • Observability: Aggregates latency, error rates, and trace data (via Langfuse).
  • Evaluation Service: Orchestrates judge inference using small language models (SLMs) running locally via Ollama.
  • Lifecycle Management: Implements a four-zone qualification cycle (Test → Human Gate → Production → Quarantine). Models are automatically quarantined if their compliance scores degrade below configurable thresholds.
• Trajectory-Based Routing: Unlike standard routing based on instantaneous performance (latency/cost), govllm incorporates score history. It can route to models with improving compliance trajectories even if their current score is lower than a declining model.

Judge Evaluation Protocol

The authors validate the framework using a ground truth corpus of 49 annotated prompt/response pairs across five regulatory criteria (Transparency, Human Oversight, Data Privacy, Non-Manipulation, Prompt Injection). These were evaluated by four SLMs (phi4-mini, mistral:7b, gemma3:4b, qwen3:1.7b) running fully on-premise. The evaluation tested three question-order conditions (original, reversed, permuted) to measure position bias and structural failure modes.

3. Key Contributions

Theoretical and Architectural

1. Governance from Metrics: Formalizes compliance as a continuous signal derived from production observability.
2. Profile-as-Jury: Proposes the computational transposition of human expert panels, where specialized LLMs evaluate specific regulatory roles.
3. Governed Qualification Lifecycle: Implements a test-human gate-production-quarantine pipeline aligned with AI Act Article 9 (continuous risk management) and Article 14 (human oversight).
4. Compliance Gate: A policy-as-code mechanism that automatically excludes models failing minimum threshold scores for specific criteria from routing.

Empirical and Diagnostic

5. Inter-Judge Variance as a Signal: Reframes disagreement among judges not as noise to be suppressed, but as a regulatory uncertainty signal warranting human arbitration.
6. Intra-Judge Incoherence Detection: Identifies and formalizes three structural failure modes in small regulatory judges that are observable without ground truth:
   • Truth Bias: A systematic default to "compliant" (true) regardless of violations.
   • Reasoning/Output Dissociation: Producing a reason that identifies a violation while simultaneously returning a compliant score.
   • Prompt Architecture Sensitivity: High sensitivity to question ordering and output format (e.g., forced-choice bias).
7. Checklist-Based Validity Assessment: Introduces a binary-checklist protocol anchored to jurisprudential sources (CNIL, ANSSI, EU AI Act) to measure judge validity independently of reliability.

4. Experimental Results

The study evaluated four SLMs (1.7B–7B parameters) on 49 cases across three question-orderings.

• Model Performance: No single model dominated across all criteria. phi4-mini (3.8B) achieved the highest global agreement (69.1% on the original order), outperforming the larger mistral:7b (51.5%). However, phi4-mini's robustness was conditional; it degraded significantly (−25 percentage points) on the data_privacy criterion under permuted question ordering.
• Specialization: A specialized panel (Profile-as-jury), assigning each criterion to its most valid judge, outperformed the best single judge by +3.5 percentage points (72.6% vs. 69.1%) and the baseline by +10.9 points.
• Position Bias: Position bias was found to be a structural property of small judges, degrading agreement by up to 25 percentage points under question-order permutation. mistral:7b showed no sensitivity to order but remained the least accurate overall.
• Failure Modes:
  • gemma3:4b exhibited low reason-consistency (69.1%), with reasons often ambiguous or contradictory to scores.
  • mistral:7b displayed an "eloquent but wrong" pattern: high internal consistency (92.7%) but low agreement with ground truth (51.5%).
  • qwen3:1.7b showed strong position sensitivity and context-dependent calibration.
• Parameter Count: Parameter count was a poor proxy for governance score (Pearson correlation r = −0.39), suggesting domain-specific calibration matters more than raw capacity in regulatory evaluation.

5. Significance and Claims

The paper claims that meaningful regulatory compliance monitoring is achievable using small, local models, provided that:

1. Judges are specialized per criterion rather than relying on a single generalist model.
2. Judges are evaluated for validity (accuracy against ground truth) rather than just reliability (consistency).
3. The governance architecture treats judge uncertainty (inter-judge variance) as a regulatory signal for human escalation, rather than a defect to be averaged out.

The authors position govllm as a concrete, reproducible, and sovereign step toward operationalizing the continuous monitoring requirements of the EU AI Act, particularly for high-risk systems where static audits are insufficient. The framework addresses the "compliance fiction" by shifting from point-in-time declarations to continuous, observable signals derived from actual production interactions, all while maintaining data sovereignty through fully on-premise execution.

The paper remains modest in its claims, noting that the ground truth corpus is preliminary, statistical significance is limited by sample size, and the framework's compliance signals are statistical estimates rather than legal determinations. Future work is identified as necessary for large-scale empirical validation of trajectory-based routing and the measurement of epistemic discrimination across model families.