Controlling Authority Retrieval: A Missing Retrieval Objective for Authority-Governed Knowledge

1. Problem Definition: Controlling Authority Retrieval (CAR)

The paper identifies a critical failure mode in standard Retrieval-Augmented Generation (RAG) systems within domains where knowledge is governed by formal authority and temporal supersession (e.g., law, drug regulation, software security).

• The Core Issue: In these domains, a later document can formally void an earlier one (e.g., a patch note voiding a vulnerability disclosure, a court ruling overruling a precedent). However, the controlling document often has semantic distance from the user query.
  • Example: A user asks, "Is the lodash prototype pollution vulnerability still unpatched?" The most semantically relevant document is the original CVE disclosure (high relevance, mentions "unpatched"). The correct answer requires the patch note (low relevance, mentions "fixed in version 4.17.12").
• Formal Definition: The authors define Controlling Authority Retrieval (CAR) as the task of recovering the active frontier of the authority closure of the semantic anchor set.
  • Let $A_k(q)$ be the top-$k$ semantically relevant documents (anchors).
  • Let $cl(A)$ be the authority closure (all documents superseded by or superseding $A$).
  • Let $front(S)$ be the active frontier (maximal elements in $S$ that are not superseded by any other document in $S$).
  • Target: $C^*_k(q) = front(cl(A_k(q)))$.
• The Gap: Standard retrieval optimizes for semantic similarity ($\arg \max s(q, d)$), whereas CAR requires traversing authority edges and checking domination. These are mathematically distinct problems.

2. Methodology and Theoretical Framework

The paper establishes a rigorous theoretical framework to characterize, bound, and solve CAR.

A. Key Metrics

• TCA (Temporal Compliance Accuracy): A new metric measuring whether the retrieved set $R$ contains the correct active answer and does not ignore any superseding documents.
  • $TCA = 1$ if: (1) The active frontier is included in $R$, and (2) No superseded document in $R$ lacks its superseder in $R$.
• Semantic-Authority Divergence ($\Delta(q)$): The difference in relevance score between the best semantic anchor and the best document in the active frontier. If $\Delta(q) > 0$, standard retrieval fails.

B. Theoretical Results (Theorems & Propositions)

1. Theorem 1 (Objective Mismatch): Proves that TCA and Recall@k are decoupled. A system can achieve Recall@k = 1 (retrieving the anchor) while TCA = 0 if the controlling document is semantically distant. This explains why larger, more powerful dense models often perform worse on CAR tasks (they align more strongly with the wrong anchor).
2. Theorem 2 (Closure Necessity): Proves that any system achieving TCA = 1 must perform proactive entity-scoped search. It cannot rely solely on query-document relevance; it must retrieve documents based on entity scope and timestamps independent of the query's initial semantic match.
3. Theorem 3 (Decomposition): The CAR problem decomposes into three necessary steps:
   • Anchor Discovery: Retrieve the semantically aligned (but potentially superseded) document.
   • Authority Resolution: Extend to the full authority chain via entity-indexed lookup.
   • Active-Status Filtering: Filter to the active frontier.
4. Theorem 4 (Correctness Characterization): Provides necessary and sufficient conditions for any retrieved set $R$ to achieve TCA = 1: Frontier Inclusion and No-Ignored-Superseder.
5. Proposition 2 (Scope Identifiability Upper Bound): Establishes a hard worst-case ceiling for any "scope-indexed" algorithm (one without direct access to authority rules).
   • $TCA@k \leq \phi(q) \cdot R_{anchor}(q)$.
   • Where $\phi(q)$ is the scope identifiability (inverse of the number of indistinguishable authority chains sharing the entity scope). If multiple chains exist, the algorithm cannot distinguish them without rule-based disambiguation.
6. Proposition 1 (Error Factorization): $TCA = R_{anchor} \cdot \phi(q) \cdot R_{frontier}$. This separates failures into: missing the anchor, scope ambiguity, and frontier recovery errors.

C. Proposed Solution: The Two-Stage Pipeline

The authors propose a Two-Stage Architecture that implements the decomposition theorem:

1. Stage 1 (Anchor Discovery): Use standard retrieval (e.g., BM25 or Dense) to find the semantically relevant anchor document.
2. Stage 2 (Authority Resolution & Filtering):
   • Extract entity identifiers (e.g., CVE ID, Drug Name) from the anchor.
   • Perform an entity-indexed lookup to retrieve all documents in that entity's scope.
   • Apply domain-specific rules (via a Retrieved-Set Supersession Graph - RSSG) to determine the active frontier and filter out superseded documents.

3. Experimental Validation

The authors validated their theory on three independent real-world corpora and synthetic benchmarks.

A. Benchmarks

1. Security (GHSA): 159 real CVE disclosure vs. patch note pairs.
2. Legal (SCOTUS): 122 Supreme Court overruling pairs.
3. Medical/Regulatory (FDA): 500 drug approval vs. recall enforcement pairs.
4. Synthetic: FinSuperQA (Compliance), CVEPATCHQA, LEGALPRECEDENTQA.

B. Key Results

• Failure of Dense Retrieval: Standard dense models (MiniLM, E5-large) and lexical models (BM25) fail catastrophically on CAR tasks when $\Delta(q) > 0$.
  • FDA: Dense TCA@5 = 0.064 (vs. Two-Stage 0.774).
  • SCOTUS: Dense TCA@5 = 0.172 (vs. Two-Stage 0.926).
  • GHSA: Dense TCA@5 = 0.270 (vs. Two-Stage 0.975).
• Scale Does Not Help: Increasing model parameters (from 22M to 335M) did not improve TCA; in some cases (E5 family), it worsened performance because larger models better aligned with the wrong (superseded) document.
• Success of Two-Stage: The proposed Two-Stage pipeline achieved TCA@5 > 0.77 across all domains, approaching the theoretical ceiling defined by scope ambiguity.
• Downstream Impact (LLM Reliability): When feeding retrieved documents to GPT-4o-mini:
  • Dense Retrieval: Caused the LLM to confidently assert "not patched" for 39% of queries where a patch existed.
  • Two-Stage: Reduced confident wrong assertions to 16.4%.

4. Significance and Contributions

1. New Retrieval Objective: The paper formalizes Controlling Authority Retrieval (CAR) as a distinct class of retrieval problems where semantic relevance is insufficient and potentially misleading.
2. Theoretical Limits: It proves that relevance-based retrieval is structurally incapable of solving CAR without architectural changes (Theorem 1 & 2) and establishes a hard upper bound based on scope ambiguity (Proposition 2).
3. Architectural Insight: It demonstrates that entity-indexed structured retrieval is the correct solution, not larger language models or better embeddings. The "Two-Stage" approach is shown to be necessary and sufficient for domains with recoverable scope.
4. Practical Impact: The work highlights a critical safety risk in RAG systems for regulated industries (law, medicine, security), where retrieving superseded information leads to confident but dangerous hallucinations.
5. Open Source: The authors released four benchmark datasets, domain adapters, and a single-command scorer to facilitate future research in authority-governed retrieval.

Conclusion

The paper argues that in authority-governed domains, the "correct" document is often the one that voids the most relevant document. Standard RAG systems fail because they optimize for semantic similarity, not temporal validity. The proposed Two-Stage Architecture, which combines semantic anchor retrieval with entity-scoped authority resolution, is the only method proven to achieve high Temporal Compliance Accuracy, effectively bridging the gap between semantic relevance and regulatory correctness.