🤖 AI

Neuro-Symbolic Agents for Hallucination-Free Requirements Reuse

This paper introduces a neuro-symbolic multi-agent system that combines Large Language Models as heuristics with a deterministic symbolic validator to enable hallucination-free, structurally valid requirements reuse within the OOMRAM framework, achieving 100% coverage and near-zero constraint violations.

Original authors: Ahmed Ibrahim

Published 2026-05-06

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Ahmed Ibrahim

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to build a custom house, but you have to choose from a massive, pre-approved catalog of blueprints. This catalog is called OOMRAM. It's like a giant, strict flowchart that says, "If you want a kitchen, you must also have a sink," and "You can pick either a red door or a blue door, but never both."

The problem is that the original way to use this catalog is like a robot that only understands exact codes. If you say, "I want a cozy home with a big garden," the robot can't help you unless you know the exact code for "cozy home."

Enter Large Language Models (LLMs). These are like super-smart, chatty architects who understand your natural language perfectly. You can tell them, "I want a cozy home," and they will suggest ideas. But there's a catch: these chatty architects are prone to hallucinations. They might invent a "flying staircase" that doesn't exist in the catalog, or they might try to put a red door and a blue door on the same house, breaking the rules.

This paper introduces a new team of workers—a Neuro-Symbolic Multi-Agent System—to fix this. Think of it as a construction crew with four specific roles:

The Navigator: A guide who looks at the catalog and says, "Okay, we need to decide on the roof next."
The Interpreter (The LLM): The chatty architect. They listen to your "cozy home" idea and suggest specific parts from the catalog (e.g., "Let's pick the slate roof").
The Validator (The Strict Inspector): This is the most important part. The Validator is not an AI; it's a rigid, mathematical rule-checker. It looks at the Interpreter's suggestion and checks it against the catalog's strict rules.
- Scenario: If the Interpreter suggests "Red Door AND Blue Door," the Validator immediately slams the gavel: "STOP! That violates the rules. You can only pick one."
- The Validator then sends the suggestion back to the Interpreter to try again.
The Scribe: Once the Inspector is happy, this person writes down the final, approved list of parts.

How it works in practice:
The system runs in a loop. The Interpreter guesses, the Validator checks, and if there's a mistake, they go back and forth until the rules are perfectly followed. Because the Validator is a strict computer program (not a guessing AI), it guarantees that the final house plan is logically sound and follows every single rule of the catalog.

The Results:
The researchers tested this on two types of "catalogs": one for digital record-keeping systems and one for smart homes.

No Broken Rules: The system achieved 100% structural validity. Every single requirement it generated followed the rules. It eliminated "hallucinations" (invented parts or illegal combinations) almost entirely.
The "F1-Score" Confusion: In standard AI tests, they measure how close the AI's answer is to a single "perfect" human answer. This system got a "moderate" score there. Why? Because for a "cozy home," there isn't just one right answer. There are hundreds of valid combinations. The system found a valid one, just not the exact one a human tester might have picked. The paper argues that finding any valid, rule-abiding solution is more important than matching a single human guess.
Speed: It took a few minutes per project, which is fast enough for real use.

The Big Takeaway:
This paper proves that you can have the best of both worlds: the flexibility of a chatty AI that understands human language, combined with the safety of a rigid rule-checker that prevents errors. By keeping the "creative" AI separate from the "strict" rule-checker, they created a system that can automatically turn vague ideas into perfectly structured, error-free technical requirements without making things up.

In short: They built a team where the creative dreamer is always supervised by a strict accountant, ensuring the final plan is both imaginative and legally sound.

Technical Summary: Neuro-Symbolic Agents for Hallucination-Free Requirements Reuse

1. Problem Statement

Requirements Engineering (RE) is inherently costly and error-prone. While reusing validated requirements from software product lines (e.g., via the Object-Oriented Method for Requirements Authoring and Management, or OOMRAM) offers a strategy to reduce effort, existing frameworks face a critical bottleneck: exact identifier matching. Traditional OOMRAM retrieval requires users to specify precise requirement IDs, preventing the automatic generation of specifications from natural-language project descriptions.

Large Language Models (LLMs) offer the linguistic flexibility to interpret vague stakeholder visions and map them to requirements. However, LLMs are prone to hallucinations, such as inventing non-existent identifiers, ignoring mandatory constraints, or combining mutually exclusive options. Current RE tools leveraging LLMs often lack formal correctness guarantees, leaving generated specifications vulnerable to logical inconsistencies and structural invalidity, which is particularly dangerous in safety-critical or highly regulated domains.

2. Methodology: A Neuro-Symbolic Multi-Agent System

The authors propose a neuro-symbolic multi-agent architecture that re-conceptualizes requirements reuse as a Model-Driven Elicitation process. The system operationalizes the formal OOMRAM lattice as a deterministic domain model, using an LLM only as a non-deterministic heuristic for traversing this model.

Core Architecture

Built on LangGraph, the system employs four specialized agents that share a typed state (AgentState):

Navigator: Simulates a depth-first traversal of the requirement lattice, selecting the next discriminant node to expand based on the project vision and current state.
Interpreter (LLM): A non-deterministic agent (using llama3.1:8b with temperature 0) that interprets the natural-language vision and proposes a set of child requirement IDs for the current node. It outputs structured JSON.
Validator (Symbolic): A deterministic, algorithmic component (pure Python, no LLM) that acts as a runtime guardrail. It checks every proposed selection against the lattice schema's Boolean satisfiability constraints ( $\Phi$ $Φ$ ). It enforces:
- Core Requirements: All mandatory nodes must be selected.
- Single Adaptor (SA/XOR): Exactly one child must be selected.
- Multiple Adaptor (MA/OR): At least one child must be selected.
- Structural Integrity: No orphaned nodes (a child cannot be selected without its parent).
- Mechanism: If a violation is detected, the Validator rejects the proposal and returns an error log to the Interpreter, forcing a revision. This loop continues until constraints are satisfied.
Scribe: Compiles the final, validated set of requirement IDs into a formatted specification.

Key Technical Strategies

Sub-graph Navigation: To handle large industrial lattices and prevent "lost-in-the-middle" reasoning failures, the system restricts the Interpreter's context to a localized subset (frontier) of the lattice at each step. This ensures constant context size ( $O(1)$ ) and linear scaling ( $O(N)$ ) with the number of decision points.
Semantic Routing: A preliminary router uses sentence-transformer embeddings to select the appropriate lattice family (e.g., eRecordKeeping, SmartHome) based on the input vision text, enabling multi-domain support without manual selection.
Formal Specification: The system defines the lattice as a Directed Acyclic Graph (DAG) $L = (V, E, T, \Phi)$ and enforces a global consistency predicate $\Psi(S)$ , ensuring the final selection set is a valid instance of the metamodel.

3. Key Contributions

Operationalizing OOMRAM for the LLM Era: This is the first work to replace exact-ID selection with LLM-driven natural-language traversal, making the formal OOMRAM lattice practical for modern RE.
Deterministic Hallucination Mitigation: The introduction of a fully deterministic Validator that acts as a hard-coded gatekeeper. By algorithmically enforcing constraints within the generation loop, the system eliminates illegal requirement combinations entirely, rather than relying on heuristic filtering.
Reproducible Autonomous Benchmark: The authors provide a benchmark demonstrating reliable traversal, near-zero constraint violations, and full structural validity across diverse application families.
Model-Agnostic Implementation: An open-source framework built on LangGraph and local Ollama, designed to scale to larger industrial lattices and ensure privacy/reproducibility.

4. Experimental Evaluation and Results

The system was evaluated in autonomous simulation mode on ten project visions drawn from two application families: the original eRecordKeeping lattice (approx. 60 requirements) and a synthetic SmartHome lattice (approx. 20 requirements). The corpus was expanded to include 12 families grounded in real-world standards (e.g., ISO 26262, ISO 20022, TM Forum eTOM).

Quantitative Results

Structural Validity: The system achieved 100% completion and 100% structural validity across all tested visions.
Constraint Violations: The final constraint-violation rate was 0.2% (1 violation detected and corrected during the process). In contrast, an ablation study removing the Validator resulted in a 5.1 average violation rate and only 14% structural validity.
F1-Score: Against manually crafted gold standards, the F1-score ranged from 0.47 to 0.81. The authors argue this moderate score is not a failure but a consequence of the vast "Valid Configuration Space" in product lines, where multiple legitimate optional/alternative choices exist.
Latency: Average wall-clock latency was 210 seconds per vision, with a mean of 78 LLM calls. The Validator added negligible runtime.

Qualitative Findings

Intentional Conflict: The system successfully navigates conflicts where the LLM attempts to satisfy stakeholder desires (e.g., "cheap system") by violating mandatory constraints (e.g., security). The Validator rejects these, forcing the LLM to find a compliant alternative (e.g., selecting the cheapest valid option).
Error Typology: Pre-validation hallucinations varied by domain (e.g., cardinality errors in HR/Payroll, structural orphans in Telecom, constraint neglect in Smart Grid), but the symbolic Validator reduced the final error rate to 0% in all cases.

5. Significance and Claims

The paper claims that the future of RE automation lies not in scaling LLM parameters, but in the harmonious coupling of neural probabilistic reasoning with hard-coded symbolic logic.

Shift in Metrics: The authors argue that for high-compliance RE, Structural Validity Rate (SVR) and Mandatory Recall are more critical metrics than traditional NLP metrics like Precision or F1-score. Achieving 100% SVR proves the system transforms stochastic generation into a formal engineering process.
Trustworthy AI: By separating the LLM's open-ended language understanding from a rule-based constraint enforcer, the system provides a trustworthy pipeline that prevents specification drift while retaining the flexibility to interpret vague visions.
Scalability: The sub-graph navigation strategy ensures the system can theoretically scale to industrial lattices without degrading reasoning quality, provided the context window remains constant.

The work concludes that while LLMs are useful for interpreting intent, they are fundamentally untrustworthy for structural logic, a task that must be offloaded to symbolic components to ensure hallucination-free requirements reuse.