💻 computer science

From Naive RAG to Deep Agentic Retrieval: An Evolving Context Engineering Pipeline for Regulatory Compliance

This paper details the evolution of a regulatory compliance retrieval system at Ontario Power Generation from naive RAG to a sophisticated multi-agent architecture called Progressive Evidence Acquisition with Cost-Aware Escalation (PEA-CAE), demonstrating that advanced context engineering and adaptive evidence acquisition are more economically viable and effective than domain-specific fine-tuning for large, evolving enterprise corpora.

Original authors: Mishca de Costa, Muhammad Saleh Anwar, Dave Mercier, Issam Hammad

Published 2026-07-29

📖 7 min read🧠 Deep dive

CC BY 4.0

Original authors: Mishca de Costa, Muhammad Saleh Anwar, Dave Mercier, Issam Hammad

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a super-smart robot friend who can read any book in the world and answer your questions instantly. This is the promise of Large Language Models (LLMs), the AI brains behind many modern chatbots. But here's the catch: if you ask this robot about a specific, complicated topic—like the rules for electricity prices in a specific province—it might get confused. Why? Because the robot's "memory" is limited, and if you dump a whole library of documents into its brain at once, it gets overwhelmed. It's like trying to find a specific needle in a haystack by shoving the entire haystack into a tiny box; the needle is technically there, but it's buried under too much junk. This is the problem of "context engineering": figuring out how to feed the robot just the right amount of information so it can think clearly without getting lost.

The big question researchers are asking is: How do we build a system that doesn't just guess, but actually searches for the right answers in a massive, constantly changing pile of legal and technical documents? The paper you're about to read explores this journey. It moves from a simple "search-and-read" method to a much smarter, multi-step detective process. The authors suggest that instead of trying to retrain the robot's brain to memorize every new law (which is expensive and slow), it's better to teach the robot how to be a better researcher. They found that the key isn't giving the robot more space to think, but giving it better tools to find the exact facts it needs, step-by-step, while keeping costs and time in check.

From "Just Ask" to "Super Detective": The Evolution of an AI Researcher

This paper tells the story of how a team at Ontario Power Generation (OPG) upgraded their AI system to handle the massive, complex task of regulatory compliance. Think of it as the story of a junior librarian who slowly learns to become a master detective.

The Starting Line: The "Naïve" Librarian

In the beginning, the team used a "Naïve RAG" system. Imagine a librarian who, when you ask a question, grabs the first 20 books that look vaguely related and shoves them all onto your desk at once. If you ask, "How much did the hydro plant cost in 2022?", the librarian might dump 25,000 words of text on you, including pages about 2019, 2023, and unrelated equipment.

The problem? The AI (the "reader") gets overwhelmed. It's like trying to find a specific sentence in a novel while someone is shouting 50 other sentences in your ear. The more text you feed it, the worse it gets at finding the right answer. This is what the authors call "context rot." The facts are there, but they get lost in the noise. The librarian just keeps adding more books, hoping the answer is in there, but it only makes the process slower and more expensive.

Level Up: The "Active" Detective

The team realized the AI needed to stop being a passive receiver and start being an active investigator. This was Stage II: Agentic Retrieval.

Now, instead of dumping a pile of books on the desk, the AI acts like a detective with a magnifying glass. It asks the librarian for just five pages at a time. It reads them, thinks, and then says, "Okay, those pages mention '2022,' but I need to check the 'Exhibit C' file specifically." It then asks for a new, more specific set of pages.

This is called progressive discovery. The AI doesn't just guess; it refines its search. If it finds a really promising lead in a single page, it might decide to read the entire document to be sure. But it's careful! It only does this full read if the "clue" is strong enough to justify the extra time and money. It's like a detective deciding whether to break into a house: only do it if the evidence suggests the criminal is definitely inside.

The Master Class: The "Deep Agent" with a Team

In Stage III, the system gets even smarter. The AI doesn't just search; it plans. Before it even starts looking, it breaks a big, scary question into a list of smaller, manageable tasks.

Imagine you need to compare the costs of two different power plants over ten years. A simple AI would get confused. But this "Deep Agent" creates a research plan:

"Find the 2020 costs for Plant A."
"Find the 2020 costs for Plant B."
"Compare them."

To handle this, the main AI hires a team of sub-agents (little helper robots). One helper reads a huge document and summarizes it into a tiny note. The main AI never sees the messy, 50-page document; it only sees the clean, 2-sentence summary. This prevents the "context rot" from happening again. The main AI stays fresh and focused, while the helpers do the heavy lifting.

Also, because of strict privacy laws in Canada, the system has a special trick. Some documents have charts and graphs that require a "vision" AI to read. Since the main AI can't send those images to a foreign server, the system has a special "local" AI in Canada that reads the images and sends back just the text. It's like having a specialized translator who stays in the room so no secrets leave the building.

The Secret Sauce: PEA-CAE

The authors formalized this smart, step-by-step approach into a framework they call PEA-CAE (Progressive Evidence Acquisition with Cost-Aware Escalation).

Think of it as a rulebook for a smart shopper:

Start small: Look at the price tags (chunks of text) first.
Don't buy the whole store: Only buy the whole item (read the full document) if the price tag looks suspiciously good.
Compress your list: Summarize your findings so you don't carry too much weight.

The paper shows that this method works. In their tests, the system made 67 "chunk" searches but only escalated to full-document reading 7 times. That's a ratio of about 10 to 1. It means the system is usually right with just a quick peek, saving a ton of time and money. When it did have to read a full document, it took between 0.7 and 12 seconds, proving that the "full read" is a heavy tool used only when necessary.

What This Means for the Future

The big takeaway from this paper is that for huge, changing libraries of rules and regulations, you don't need to retrain the AI's brain. You don't need to teach it every new law. Instead, you need to build a better search engine around it.

The authors argue that "context engineering"—the art of feeding the AI the right amount of information at the right time—is the real magic. It's better than trying to force the AI to memorize everything. By using a system that plans, delegates, and escalates carefully, companies can get accurate, traceable answers from their AI without it getting lost in a sea of text.

In short, the paper suggests that the future of AI in complex fields isn't about making the robot smarter in a vacuum; it's about giving it a better map, a sharper magnifying glass, and a team of helpers to do the heavy lifting. It's about teaching the robot to be a detective, not just a reader.

Technical Summary: From Naïve RAG to Deep Agentic Retrieval for Regulatory Compliance

1. Problem Domain and Motivation

The paper addresses the deployment of Large Language Models (LLMs) within highly regulated industries, specifically focusing on Ontario Power Generation's (OPG) need to analyze thousands of pages of regulatory filings for the Ontario Energy Board (OEB). The core challenge is not merely answering questions but engineering a system that reliably, efficiently, and verifiably retrieves authoritative context from a large, proprietary, and continuously evolving corpus.

The authors identify a critical failure mode in standard implementations: Context Rot. They argue that context length is not a free resource. As context windows expand with irrelevant tokens, attention competition degrades answer quality, and retrieval signals weaken. Consequently, the objective shifts from maximizing raw context volume to maximizing relevant evidence density.

The paper explicitly rejects domain-specific fine-tuning as the primary solution for this use case. Fine-tuning is deemed economically mismatched for evolving corpora where knowledge freezes upon training, whereas regulatory filings are frequently updated. Instead, the authors advocate for Context Engineering—the deliberate construction of high-signal inference-time context through retrieval, filtering, and compression.

2. Methodological Evolution

The paper traces the evolution of the retrieval pipeline through three distinct stages, each motivated by specific failure modes of the previous iteration.

Stage I: Naïve RAG Baseline

Architecture: A standard pipeline involving OCR/structured extraction, chunking (500–800 tokens), embedding, and hybrid retrieval (dense vector + keyword) with cross-encoder re-ranking.
Limitations: The system is deterministic and stateless. If the initial retrieval fails to surface relevant chunks due to vocabulary mismatch or multi-hop dependencies, the LLM cannot self-correct. The only mitigation is increasing the number of retrieved chunks ( $k$ ), which linearly increases cost and latency while exacerbating context rot.
Outcome: While functional for simple lookups, it lacks the flexibility to handle complex, multi-hop regulatory queries.

Stage II: Agentic Retrieval via Function Calling

Architecture: The LLM transitions from a passive consumer to an active retriever. It is equipped with a search tool and iteratively reformulates queries based on intermediate results (pseudo-relevance feedback).
Key Mechanisms:
- Progressive Discovery: The agent issues multiple tool calls with refined queries, narrowing the search space using metadata filters (e.g., specific exhibit IDs or rate case years).
- Chunk-to-Document Escalation: A decision rule is introduced to escalate from chunk-level retrieval to full-document reading only when the expected evidence gain ( $\Delta U$ ) justifies the added latency ( $\Delta T$ ) and cost ( $\Delta C$ ).
- Capability Partitioning: To satisfy Canadian data residency constraints, visual content (charts/graphs) requiring vision inference is offloaded to a Canada-hosted model, while the main reasoning agent remains separate.
Limitations: Function calling introduces JSON schema overhead. In worst-case scenarios with many low-quality queries, token consumption can exceed the Stage I baseline.

Stage III: Deep Agentic Retrieval (PEA-CAE)

Architecture: A mature system featuring explicit planning, hierarchical subagent delegation, and code-based tool synthesis.
Key Mechanisms:
- Explicit Planning: The agent decomposes complex queries into a sequence of sub-questions before retrieval begins, storing intermediate findings in working memory.
- Hierarchical Subagents: To prevent context rot in long reasoning chains, the main agent spawns subordinate agents for bounded sub-tasks. Subagents perform deep research and return only compressed summaries, preventing intermediate retrieval artifacts from consuming the main agent's context window.
- Code-Based Tool Synthesis: Instead of pre-defined JSON functions, the agent writes Python/Bash code at runtime to interact with APIs and process data. This amortizes tool overhead and allows for combinatorial expressiveness without a fixed tool limit.
- PEA-CAE Framework: The system is formalized as Progressive Evidence Acquisition with Cost-Aware Escalation. It enforces rules to filter first, compress early, escalate late, and compress breadth.

3. Key Contributions

The paper makes five primary contributions:

Algorithmic Characterization: A stage-by-stage analysis of three retrieval architectures deployed in production, linking specific failure modes to architectural transitions.
Formal Framework (PEA-CAE): A formalization of the mature retrieval policy that balances evidence acquisition with cost and latency constraints.
Empirical Analysis: Operational trace data demonstrating the tradeoffs between context window utilization, retrieval precision, and escalation frequency.
Data Sovereignty Pattern: A capability partitioning strategy for multi-jurisdiction deployments, separating visual processing (hosted locally) from general reasoning.
Theoretical Connection: A comparative analysis linking practical system design (adaptive query reformulation, progressive discovery) to classical information retrieval and information foraging theories.

4. Results and Observations

Based on operational traces from the Stage III (Deep Agent) system:

Success Rate: In a sample of 14 runs, all 6 OEB agentic runs were successful.
Latency: Task durations ranged from 24.8 seconds to 223 seconds (median 92.1s), reflecting a sustained research loop rather than a single-pass lookup.
Escalation Efficiency: The system demonstrated a search-to-escalation ratio of approximately 9.6:1. Full-document extraction occurred in only ~10% of retrieval events, confirming the "selective escalation" design intent.
Latency Variance: Full-document extraction showed high tail latency (up to 12.14s), reinforcing the necessity of keeping escalation selective to avoid performance degradation.

5. Significance and Claims

The paper claims that for large, evolving regulatory corpora, context engineering is a more tractable and economically viable path than domain-specific fine-tuning. The "intelligence" of the system resides not in the model weights but in the algorithms that construct high-signal context windows.

The authors argue that the progression from naïve RAG to deep agentic retrieval mirrors classical information retrieval ideas, introducing practical system primitives such as:

Adaptive query reformulation.
Progressive document discovery.
Hierarchical subagent summarization.

The central thesis is that context remains a scarce resource even as nominal window sizes grow. Therefore, the optimal strategy is to retrieve progressively, escalate selectively, and compress aggressively. The paper concludes that operational success in regulated AI depends less on model adaptation and more on the joint management of retrieval quality, tool overhead, latency, and context growth.