Retrieval-Augmented Large Language Models as Components of Cognitive Computing architecture for Regulatory Knowledge Management

Technical Summary: Retrieval-Augmented Large Language Models as Components of Cognitive Computing Architecture for Regulatory Knowledge Management

Problem Statement
The paper addresses the limitations of deploying standalone Large Language Models (LLMs) in high-stakes regulatory and legal environments. While LLMs offer advanced text processing, they suffer from "hallucinations" (generating content ungrounded in sources), lack of traceability, and an inability to dynamically update knowledge without retraining. Furthermore, reliance on cloud-based LLM services poses significant risks regarding data sovereignty, privacy, and the security of sensitive legal or business information. The author identifies a gap between innovative RAG (Retrieval-Augmented Generation) research and its practical application in creating epistemically reliable cognitive computing components for regulatory management, particularly within on-premises environments constrained by consumer-grade hardware.

Methodology
The study proposes and validates a hybrid cognitive architecture where locally deployed LLMs function as semantic processing modules, augmented by an external RAG layer for controlled knowledge retrieval.

• Hardware and Environment: The implementation utilized consumer-class hardware (Apple M4 processor, 24 GB unified memory) running macOS. No high-end GPU accelerators were used.
• Models: Two Polish language models were tested: Bielik (1.5B parameters) and PLLuM (12B parameters).
• RAG Architecture:
  • Runtime: Ollama and LM Studio were used as execution environments. LM Studio served as the primary interface, integrated with the Model Context Protocol (MCP).
  • Retrieval Mechanisms: The study employed two distinct retrieval strategies:
    1. Vector-based: Using ChromaDB for local vector storage and BERT-based embeddings for semantic search.
    2. Tool-based: Integrating Zotero (via MCP) for structured access to curated bibliographic databases and Tavily (via MCP) for web-based legal resource fetching (e.g., Internetowy System Aktów Prawnych).
  • Orchestration: LangChain was used for prompt orchestration, and vLLM for local inference serving.
• Experimental Design: The research involved two rounds of testing on four specific research questions regarding the definition and regulation of "real estate" in Polish law:
  1. Round 1: Baseline generation using standalone LLMs.
  2. Round 2: Generation using LLMs augmented with RAG (retrieval, extension, and generation phases).
• Evaluation: Generated texts were analyzed for grammatical correctness, readability, and stylistic properties using Jasnopis.pl (readability) and Logios.dev (syntactic analysis, Gunning FOG Index, and Plain Language Index). Substantive correctness was manually verified against legal sources.

Key Results

• Epistemic Reliability: The integration of RAG significantly improved the factual consistency and normative precision of the generated texts. The RAG-augmented models successfully reduced the risk of unsupported content generation by grounding outputs in retrieved legal acts.
• Linguistic Characteristics:
  • Specialization: RAG variants demonstrated a higher density of specialist terminology and sophisticated words (e.g., Bielik+RAG showed a 5.5% share of sophisticated words vs. 3.0% in the baseline).
  • Complexity: The Gunning FOG Index increased for RAG variants (e.g., Bielik+RAG reached 15.48), indicating higher text complexity. The authors argue this is a desirable feature for legal texts, reflecting necessary terminological precision rather than a defect.
  • Style: PLLuM+RAG exhibited longer sentences and paragraphs, aligning with the enumerative style of legal regulations, while Bielik+RAG showed a more normative, adjective-heavy style.
• Substantive Errors: Despite improvements, the study identified critical errors in the PLLuM+RAG output. Specifically, the model incorrectly stated that real estate could be "movable" if it were a component part, contradicting fundamental civil law principles regarding the distinction between movable and immovable property. Additionally, the model failed to distinguish between the civil law definition of real estate and functional definitions used in administrative or tax laws, leading to methodological confusion.
• Auditability: The RAG architecture successfully introduced traceability, allowing users to verify the source of information (specific legal acts) without retraining the underlying model.

Key Contributions

1. Architectural Validation: The paper demonstrates that locally deployed LLMs, when combined with RAG on consumer-grade hardware, can function as viable components of a cognitive computing infrastructure for regulatory management.
2. Hybrid Cognitive Model: It proposes a separation of concerns where the LLM acts as a semantic interpretation module, while the RAG layer handles knowledge retrieval, contextualization, and source traceability.
3. On-Premises Feasibility: The study validates that high-quality regulatory analysis is achievable without cloud dependencies or enterprise-grade GPUs, preserving data sovereignty and enabling dynamic updates to legal repositories.
4. Critical Assessment of LLMs in Law: The research highlights that while RAG improves factual grounding, it does not eliminate the risk of conceptual errors or the misattribution of definitional authority to legal acts, underscoring the need for human oversight.

Significance and Claims
The paper claims that augmenting LLMs with RAG transforms them from "closed production-ready tools" into "semantic processing modules" within cognitive computing infrastructures. This transformation is significant for organizations operating in environments with high legal volatility, as it enables:

• Dynamic Knowledge Management: The ability to update regulatory information instantly by modifying the document repository rather than retraining the model.
• Enhanced Control: Full control over data processing, ensuring compliance with data sovereignty and privacy regulations.
• Complementary Role: The authors conclude that LLMs should not be viewed as replacements for expert knowledge or symbolic AI (Cognitive Computing) but as complementary components. They serve as effective interfaces for natural language processing and information synthesis, provided they are integrated into a hybrid architecture that includes memory, control, and validation mechanisms.

The study maintains a modest stance, acknowledging that while RAG improves reliability, the models still lack true cognitive understanding and can produce fundamental legal errors, necessitating their use as decision-support tools rather than autonomous decision-makers.