Automating the Detection of Requirement Dependencies Using Large Language Models

1. Problem Statement

Modern software systems involve complex, high-volume requirements that are inherently interconnected through various dependency types (e.g., Requires, Implements, Conflicts). Identifying these dependencies is critical for impact analysis, consistency checking, and release planning. However, the task is currently:

• Manual and Error-Prone: Relies heavily on human effort, which is time-consuming and cognitively demanding.
• Challenged by Ambiguity: Natural Language (NL) requirements often contain ambiguity and inconsistent terminology.
• Limited by Existing Tools:
  • Retrieval-based approaches rely on fixed vector representations and fail to capture domain-specific context.
  • Knowledge-based approaches (ontologies) require extensive manual maintenance.
  • Machine Learning (ML) approaches suffer from data scarcity, class imbalance (independent requirements vastly outnumber dependent ones), and a lack of inferential reasoning capabilities.
• Unexplored Potential: While Large Language Models (LLMs) excel at NL processing, their effectiveness in automating requirement dependency detection has not been rigorously explored.

2. Methodology: LEREDD Framework

The authors propose LEREDD (LLM-Enabled REquirement Dependency Detection), an automated approach that combines Retrieval-Augmented Generation (RAG) and In-Context Learning (ICL) to detect dependencies between NL requirement pairs.

Core Architecture

The framework operates in a two-phase pipeline:

1. Knowledge Retrieval Phase:
   • Contextual Retrieval (RAG): Extracts domain-specific information from the Software Requirement Specification (SRS) document. It uses a fixed-size chunking strategy (retrieving the top 10 semantically similar chunks) to provide the LLM with system architecture and component interaction context.
   • Dynamic Examples Retrieval (ICL): Retrieves semantically similar requirement pairs from an annotated dataset to serve as few-shot examples. It uses SBERT embeddings and a specific similarity aggregation formula (taking the maximum similarity between target and example pairs) to select the top 4 examples per dependency type.
2. Dependency Inference Phase:
   • An LLM (specifically GPT-4.1, identified as the best performer) processes the requirement pair augmented with the retrieved context and examples.
   • The prompt includes formal definitions of 7 dependency types: Requires, Implements, Conflicts, Contradicts, Details, Is similar, and Is a variant.
   • Output: The model generates a dependency classification, a rationale (explanation), and a confidence score (0–5 Likert scale) to encourage self-reflection and reduce hallucinations.

Experimental Setup

• Dataset: A newly curated, open-source dataset of 813 manually annotated requirement pairs across three distinct automotive systems (Traffic Jam Assist, Automated Parking Assist, Adaptive Driving Beam).
• Baselines: Compared against two State-of-the-Art (SOTA) approaches:
  1. TF-IDF & LSA: A retrieval-based lexical/semantic approach.
  2. Fine-tuned BERT: A supervised ML approach.
• Evaluation Scenarios:
  • Intra-dataset: Training and testing on the same system.
  • Cross-dataset: Training on one system and testing on another (simulating real-world scenarios where annotated data for the target system is unavailable).

3. Key Contributions

1. LEREDD Framework: The first comprehensive LLM-based approach for automated requirement dependency detection that leverages both RAG for domain grounding and ICL for task-specific guidance.
2. Novel Dataset: Release of an annotated corpus of 813 requirement pairs across three industrial-grade automotive systems, addressing the scarcity of public datasets for this task.
3. Comprehensive Evaluation: A rigorous empirical study comparing LEREDD against SOTA baselines (TF-IDF/LSA and Fine-tuned BERT) and various LLM prompting strategies (Zero-shot, Few-shot, RAG).
4. Insights on Prompting: Identification of optimal configurations, specifically that 4 examples per type, SBERT embeddings, Euclidean distance, and Max-similarity aggregation yield the best results.

4. Key Results

• Overall Performance: LEREDD achieved an average Accuracy of 92.66% and an F1 Score of 84.33% across all systems and dependency types.
• No-Dependency Filtering: LEREDD excels at identifying non-dependent pairs (the majority class), achieving an F1 score of 0.96. This allows for the automatic filtering of irrelevant pairs, significantly reducing manual analysis effort.
• Fine-Grained Dependencies:
  • LEREDD significantly outperformed baselines on difficult classes like Requires.
  • It showed relative F1 gains of 94.87% and 105.41% over baselines for the Requires dependency.
  • Zero-shot LLMs struggled with fine-grained types (average F1 ~0.39), proving that task-specific guidance (RAG + ICL) is essential.
• Cross-Dataset Robustness:
  • LEREDD: Maintained high performance (Avg F1 ~0.76) when tested on systems different from its training/example pool, showing strong generalization.
  • Fine-tuned BERT: Suffered a significant performance drop (Avg F1 dropped from 0.52 to 0.31) in cross-dataset settings due to reliance on specific training distributions.
• Efficiency: LEREDD was faster than fine-tuned BERT (1m 48s vs 4m 3s per system) while offering superior accuracy.

5. Significance and Implications

• Industrial Applicability: By effectively filtering out non-dependent pairs (which constitute the vast majority of requirement relationships), LEREDD drastically reduces the cognitive load and time required for requirements engineers.
• Overcoming Data Scarcity: The RAG+ICL approach allows the system to perform well even when annotated data for the specific target system is unavailable, a common real-world constraint where ML models typically fail.
• Reasoning over Pattern Matching: The results demonstrate that dependency detection requires inferential reasoning and domain context, which LLMs with RAG provide, whereas traditional ML models rely too heavily on statistical pattern recognition.
• Future Research: The paper establishes a new benchmark and dataset, paving the way for future work on detecting indirect/implicit dependencies and using predicted dependencies for impact analysis.

In conclusion, LEREDD represents a significant advancement in Requirements Engineering, proving that combining LLMs with retrieval strategies is a viable, robust, and highly accurate solution for automating complex dependency detection tasks.