🤖 AI

DomAgent: Leveraging Knowledge Graphs and Case-Based Reasoning for Domain-Specific Code Generation

This paper introduces DomAgent, an autonomous coding agent that leverages a novel retrieval module combining knowledge-graph reasoning and case-based reasoning to significantly enhance domain-specific code generation, enabling smaller open-source models to rival large proprietary LLMs in specialized tasks.

Original authors: Shuai Wang, Dhasarathy Parthasarathy, Robert Feldt, Yinan Yu

Published 2026-03-24

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Shuai Wang, Dhasarathy Parthasarathy, Robert Feldt, Yinan Yu

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Problem: The "Generalist" vs. The "Specialist"

Imagine you hire a brilliant, world-famous chef (a Large Language Model or LLM) to cook a very specific regional dish, like a traditional Swedish Surströmming sandwich or a complex truck engine repair manual.

The chef is amazing at cooking general meals. They know how to make pasta, steak, and cakes because they have read millions of cookbooks (public internet data). However, they have never worked in a specific truck factory before. They don't know the specific tools used there, the secret codes for the engine sensors, or the company's weird way of writing instructions.

If you ask this chef to fix a truck engine, they might guess. They might use the wrong wrench or mix up the oil with the fuel. In the world of coding, this is called Domain-Specific Code Generation. The AI is smart, but it lacks the "insider knowledge" needed for specialized jobs like truck software or medical data analysis.

The Solution: Meet "DomAgent"

The authors of this paper built a new system called DomAgent. Think of it not as a single chef, but as a Super-Intern who works alongside the chef.

This intern has two superpowers that help the chef cook the perfect truck-engine dish:

The Encyclopedia (Knowledge Graph): A structured map of all the tools and rules.
The Photo Album (Case-Based Reasoning): A collection of past successful repairs to learn from.

Here is how DomAgent works, step-by-step:

1. The "Top-Down" Map (Knowledge Graph)

Imagine the chef needs to know how to use a specific tool called read_can_signal.

Without DomAgent: The chef guesses what the tool does based on its name.
With DomAgent: The intern pulls out a Knowledge Graph. Think of this as a giant, organized family tree of the truck's software. It explicitly says: "Hey, this tool connects to the 'OverspeedWarning' sensor, and here are the exact rules for how it works."
The Analogy: It's like giving the chef a blueprint of the truck's engine instead of just letting them guess how the parts fit together.

2. The "Bottom-Up" Photo Album (Case-Based Reasoning)

Knowing the rules isn't enough; the chef needs to see how others did it before.

Without DomAgent: The chef tries to remember a similar recipe from a different country.
With DomAgent: The intern flips through a Photo Album of past successful code. But here's the trick: The intern doesn't just pick random photos. They use the Knowledge Graph to find the perfect photo.
The Analogy: If the task is "fix the brake light," the intern doesn't show a picture of fixing the radio. They find a photo of someone fixing a brake light on a similar truck model. This is called Case-Based Reasoning.

3. The "Smart Review" (The Agent's Brain)

This is the most clever part. Sometimes, the intern might grab the wrong photo or the wrong rule.

The Process: Before the chef starts cooking, the intern (the AI Agent) looks at the rules and the photos, thinks for a moment, and says: "Wait, this photo is for a 2020 model, but we are fixing a 2024 model. Let's swap it for this one."
The Analogy: It's like a Quality Control Inspector who reviews the chef's ingredients before the cooking starts, throwing out the bad ones and keeping only the best ones. This "review" step ensures the final code is perfect.

Why is this a Big Deal?

The paper tested this system in two ways:

The Test Exam (DS-1000): A standard test for data science coding.
The Real World (Truck Factory): A real job at Volvo, dealing with actual truck sensors.

The Results:

Small Models became Big Models: They used a small, open-source AI (like a junior chef) and made it perform almost as well as the massive, expensive, proprietary AI (like a celebrity chef).
Real-World Success: In the truck factory, the system went from being terrible (39% success) to being nearly perfect (96% success) just by adding this "Intern" system.
Cost Effective: You don't need to retrain the whole AI (which is like sending the chef back to culinary school for 5 years). You just give them the right map and the right photo album.

Summary Metaphor

Imagine you are trying to navigate a city you've never visited.

Standard AI: Gives you a generic map of the whole world. You get lost.
DomAgent: Gives you a GPS (Knowledge Graph) that shows the exact streets, AND a Travel Blog (Case Base) with photos of exactly how to turn at the tricky intersections.
The Result: Even if you are a new driver (a small AI), you can drive through the city perfectly without getting lost.

In short: DomAgent bridges the gap between "smart but generic" AI and "specialized but human" expertise by giving the AI the right tools and the right examples to look at before it starts working.

1. Problem Definition

The paper addresses the significant performance gap Large Language Models (LLMs) face when transitioning from general-purpose code generation to domain-specific code generation.

The Challenge: While LLMs excel at general tasks, they often fail in specialized industrial scenarios (e.g., truck software development, data science libraries) because:
1. Lack of Domain Knowledge: They are unaware of proprietary third-party libraries, specific APIs, or internal system constraints (e.g., utils.can.read() in truck CAN systems).
2. Insufficient Contextual Adaptation: They struggle to adapt generic code patterns to specific workflow requirements and mapping logic unique to a domain.
Limitations of Existing Solutions:
- Fine-tuning: High cost, difficult to maintain with rapidly changing libraries, and prone to catastrophic forgetting.
- Standard RAG (Retrieval-Augmented Generation): Often relies on plain text similarity (BM25 or embeddings), which may miss structural dependencies or retrieve irrelevant long contexts that confuse the model.
- Static Few-Shot Prompting: Fixed examples fail to cover diverse scenarios and require labor-intensive manual curation.

2. Methodology: The DomAgent Framework

The authors propose DomAgent, an autonomous coding agent that bridges the gap by integrating Top-Down Knowledge Graph (KG) reasoning with Bottom-Up Case-Based Reasoning (CBR).

A. Core Components

DomRetriever (Novel Retrieval Module):
- Top-Down Retrieval (Knowledge Graph): Uses a KG where nodes represent packages, functions, and their relationships. Given a task, an LLM classifies relevant packages and retrieves structured knowledge (descriptions, parameters) via semantic similarity.
- Bottom-Up Retrieval (Case-Based): Retrieves relevant code examples. Instead of random selection, it uses the KG to guide selection (see below).
- Iterative Reasoning & Filtering: The LLM acts as an agent that can call SearchKG and SearchCase tools. It reviews retrieved content, filters out superficially similar but functionally irrelevant information, and synthesizes a refined context before generating code.
KG-Guided Hierarchical Case Selection:
- To solve the "labor-intensive" problem of building case bases, the authors propose a method to construct a small, representative case base from a large pool.
- Process:
  - Package-Level Anchoring: Identify root nodes (packages) in the KG.
  - Function-Level Clustering: Cluster child nodes (functions) within packages using $k$ -means on semantic embeddings.
  - Coverage-Driven Selection: Iteratively select code cases that maximize the coverage of distinct packages ( $\alpha$ ) and functional clusters ( $\beta$ ).
- Result: This ensures the case base is diverse and covers the domain structure efficiently, achieving near-full performance with only 30% of the data compared to random sampling.
Agent Training:
- Supervised Fine-Tuning (SFT): Trains the LLM to generate special tokens (<search_kg>, <search_case>) to trigger retrieval tools.
- Reinforcement Learning (RL): Uses a reward model (trained on a powerful LLM) to provide feedback. The agent is rewarded for retrieving the most relevant knowledge and cases and filtering out noise, encouraging autonomous reasoning.

B. Workflow

Input: Domain-specific task query.
Reasoning Phase: The LLM invokes tools to search the KG for domain knowledge and the Case Base for examples.
Review Phase: The LLM analyzes the retrieved items, discarding irrelevant ones and refining the context.
Generation Phase: The final prompt (Task + Refined Knowledge + Refined Cases) is fed to a code generation LLM to produce the target code.

3. Key Contributions

First Integrated Agent: The first agent system specifically designed for domain-specific code generation that unifies structured knowledge (KG) and experiential reasoning (CBR) into a single bidirectional framework.
KG-Guided Case Selection: A novel method to construct compact, high-coverage case bases by exploiting structural relations in KGs, reducing the need for massive manual curation.
DomRetriever Module: A flexible, plug-and-play retrieval module that can be integrated with any LLM pipeline, enabling dynamic, iterative retrieval and synthesis.
Real-World Validation: Successful deployment in a complex industrial setting (truck software development), demonstrating practical viability beyond academic benchmarks.

4. Experimental Results

The system was evaluated on two datasets:

DS-1000 (Data Science Benchmark): 1,000 tasks across 7 major libraries (NumPy, Pandas, PyTorch, etc.).
Truck CAN Signal Dataset: 776 real-world tasks involving reading/writing CAN signals for truck control systems.

Key Findings:

Performance Gap Closure: DomAgent allows small open-source models (e.g., Qwen2.5-7B, LLaMA3.1-8B) to close the performance gap with large proprietary models (GPT-4o).
- DS-1000: DomAgent (Qwen2.5-7B) achieved 39.2% pass@1, outperforming the vanilla Qwen2.5-7B (29.3%) and approaching GPT-4o (51.0%). When paired with GPT-4o for generation, it reached 58.6%.
- Truck Domain: DomAgent (Qwen2.5-7B) achieved 96.64% pass@1, nearly matching GPT-4o with DomRetriever (98.04%) and vastly outperforming the vanilla Qwen2.5-7B (39.62%).
Ablation Studies:
- Both KG (Top-Down) and CBR (Bottom-Up) components are essential.
- The DomAgent reasoning loop (filtering/reviewing) outperforms standard RAG pipelines.
- KG-Guided Selection significantly outperforms random sampling, especially at low sampling rates (30% data usage).
Efficiency: The system demonstrates that small models equipped with structured retrieval and reasoning can outperform larger models relying solely on pre-trained weights or static prompts.

5. Significance and Impact

Industrial Applicability: The paper proves that LLMs can be effectively deployed in safety-critical, domain-specific industries (like automotive) without the prohibitive cost of fine-tuning massive models.
Cost-Effectiveness: By using small open-source models as the "brain" for retrieval and reasoning, and optionally a larger model only for final code generation, organizations can reduce inference costs while maintaining high accuracy.
Scalability: The KG-guided case selection method offers a scalable solution for maintaining code bases in rapidly evolving domains, ensuring high coverage with minimal manual effort.
Future Direction: The work highlights the potential of combining symbolic knowledge (KGs) with neural reasoning (LLMs) to solve complex, real-world engineering problems.