🤖 AI

Helicase: Uncertainty-Guided Supply Chain Knowledge Graph Construction with Autonomous Multi-Agent LLMs

This paper introduces Helicase, an autonomous multi-agent LLM system that constructs uncertainty-aware supply chain knowledge graphs by decomposing complex queries into iterative investigation plans and tracking confidence across action, trajectory, and memory layers, evaluated on the new SCQA benchmark.

Original authors: Yunbo Long, Haolang Zhao, Ge Zheng, Alexandra Brintrup

Published 2026-05-27

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Yunbo Long, Haolang Zhao, Ge Zheng, Alexandra Brintrup

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to solve a massive, real-world mystery: "Which specific parts in a Tesla car use lithium mined from Australia?"

If you ask a standard AI chatbot this question, it might guess based on what it read in its training books years ago, or it might just say, "I don't know." It can't go out and check the latest news, read a PDF report from a mining company, and cross-reference it with a battery manufacturer's website to build a new answer.

This paper introduces Helicase, a new type of AI team designed to solve these complex supply chain mysteries. Here is how it works, explained simply:

1. The Problem: The "Jigsaw Puzzle" of Information

Supply chain information is like a giant jigsaw puzzle where the pieces are scattered across the internet. Some pieces are easy to find (like a product's ingredient list on a website), but others are hidden in dusty PDF reports, obscure trade journals, or buried in social media posts.

Old AI: Tries to guess the picture based on memory. It often gets it wrong or hallucinates (makes things up).
The Challenge: You can't find the answer in just one document. You have to connect the dots between a mine in Australia, a chemical refiner in China, a battery factory in Nevada, and the final car.

2. The Solution: A Team of Specialized Detectives

Helicase isn't just one AI; it's a team of autonomous agents working together like a detective squad.

The Planner (The Captain): This agent looks at the big question and breaks it down. It decides, "First, let's find the mines. Then, let's find who buys from them."
The Search Agents (The Scouts): These agents go out and search the web. They read websites, PDF reports, and spreadsheets. They don't just read once; they send out multiple scouts to check the same fact to make sure they agree.
The Reasoning Agents (The Analysts): These agents take the clues the scouts found and figure out how they connect. "If Mine A sells to Refiner B, and Refiner B sells to Factory C, then the lithium flows from A to C."
The Coding Agents (The Cartographers): These agents draw the map. They build a Knowledge Graph—a structured diagram showing exactly who is connected to whom.

3. The Secret Sauce: "Uncertainty" as a Compass

The most important part of Helicase is that it knows what it doesn't know.

Imagine you are building a map. If you find a fact in a government report, you mark it with a Green Flag (High Confidence). If you find a fact in a random blog post that no one else mentions, you mark it with a Red Flag (High Uncertainty).
Helicase uses a "Three-Layer Uncertainty System":
1. Action Layer: "Did the search agents agree on this fact?"
2. Trajectory Layer: "Are we just going in circles repeating the same search, or are we finding new clues?"
3. Memory Layer: "Looking at the whole map we've built so far, how confident are we in the final picture?"

If the AI is unsure, it doesn't guess. It goes back, sends out more scouts, and tries to find more evidence to turn that Red Flag into a Green one. It keeps doing this until the map is clear enough or it runs out of new places to look.

4. The Test: The "SCQA" Exam

To prove it works, the authors created a test called SCQA (Supply Chain Query Assessment). It's like a final exam with 80 difficult questions.

The Easy Questions: "Does this shampoo have this ingredient?" (Answer is on the bottle).
The Hard Questions: "Which of our competitors' products use a specific chemical that is running out of supply?" (Answer requires connecting 5 different hidden sources).

The Results:

Standard AI models (like the ones you might chat with daily) failed miserably on the hard questions. They couldn't build the map or admit when they were guessing.
Helicase succeeded. It built the maps, connected the dots, and told the user exactly how confident it was in each connection. It was the only system that could handle the "low visibility" questions where the answer isn't written down anywhere single place.

5. Why It Matters

Think of Helicase as a self-driving research assistant.

Instead of a human spending weeks manually clicking through websites, reading PDFs, and drawing charts to find a supply chain risk, Helicase does it in minutes.
It doesn't just give you a "Yes" or "No." It gives you a verified map with a confidence score. If the map says, "We are 90% sure this supplier is connected to that mine," a business manager can trust that number. If it says, "We are only 40% sure," the manager knows to double-check that specific link before making a decision.

In short, Helicase turns the chaotic, messy internet into a clear, reliable, and self-checking map of the world's supply chains.

Technical Summary: HELICASE – Uncertainty-Guided Supply Chain Knowledge Graph Construction with Autonomous Multi-Agent LLMs

1. Problem Statement

Current Large Language Model (LLM) systems and multi-agent frameworks face critical limitations when applied to supply chain intelligence. While effective for general knowledge retrieval, they fail to address structural inference problems where answers are not contained in a single document but must be synthesized across fragmented, heterogeneous web resources.

Specifically, existing systems suffer from four fundamental flaws:

Lack of True Autonomy: They rely on rigid, predefined workflows or pre-processed data, unable to dynamically determine search depth or direction for ambiguous, complex queries.
Absence of Verifiable Uncertainty: They produce narrative summaries with unverified citations and hallucinations, lacking mechanisms to quantify confidence or trace reliability to source quality.
Black-Box Reasoning: The decision-making process is opaque, preventing auditability required for strategic planning.
Inability to Construct Structural Models: They lack the capacity to build intermediate structured models (e.g., Knowledge Graphs) necessary for analyzing complex systems, limiting them to narrative synthesis.

The core challenge is to answer questions like "Which Tesla components use lithium from Australian mines?" which require multi-hop reasoning across low-visibility data (e.g., regulatory filings, trade press, unstructured social signals) and demand calibrated confidence in the results.

2. Methodology: The Helicase System

Helicase is an autonomous multi-agent LLM system designed to iteratively "unwind" opaque supply chain queries into structured, uncertainty-annotated knowledge graphs (KGs). It operates via a helical process involving query enrichment, multi-source evidence harvesting, and graph construction.

2.1 Agent Architecture

The system employs a specialized multi-agent hierarchy:

Planner Agent: Decomposes high-level queries into executable investigation plans. It dynamically generates new actions targeting high-uncertainty regions of the current KG based on feedback from previous iterations.
Web Search Agents: Perform parallel, multi-source evidence retrieval from diverse modalities (HTML, PDFs, CSVs, social platforms). They utilize a consensus mechanism to score evidence quality.
Reasoning Agents: Synthesize harvested evidence, perform cross-source inference, and identify necessary structural updates.
Coding Agents: Translate reasoning decisions into deterministic, auditable JSON mutations (creating, merging, or revising nodes/edges) on the knowledge graph, utilizing fuzzy entity matching to prevent duplicates.

2.2 Three-Layer Uncertainty Quantification

A core innovation is the three-layer uncertainty framework that tracks uncertainty at different scales to guide the system and calibrate confidence:

Action-Layer: Measures the reliability of individual agent executions (e.g., factual consensus across $n$ parallel search queries). It uses LLM-based consensus scoring to distinguish factual agreement from stylistic variation.
Trajectory-Layer: Detects redundancy by computing semantic similarity between current actions and all prior actions. High redundancy signals diminishing returns, prompting the planner to pivot or terminate.
Memory-Layer: Consolidates per-fact uncertainties stored in the KG. It employs multiplicative accumulation for existing facts (where repeated independent evidence reduces uncertainty) and minimum-based initialization for new facts. The system-wide memory uncertainty ( $U_{memory}$ ) drives convergence detection.

2.3 Iterative Execution and Convergence

The system operates in a closed loop where the current KG and its uncertainty landscape feed back to the planner.

Action Selection: Candidates are prioritized based on estimated uncertainty reduction per unit cost.
Convergence: The system terminates when memory-layer uncertainty exhibits sustained stagnation (relative change below a threshold for $P$ consecutive iterations), ensuring the investigation has exhausted its ability to reduce uncertainty.

3. The SCQA Benchmark

To evaluate autonomous reasoning across the full complexity spectrum, the authors introduce SCQA (Supply Chain Query Assessment), a benchmark of 80 queries organized into four quadrants based on two dimensions:

Reasoning Complexity: Single-hop vs. Multi-hop.
Information Visibility: High-visibility (structured, public sources) vs. Low-visibility (fragmented, unstructured, niche sources).

The four quadrants are:

Q1: Single-hop, High-visibility (e.g., "Does Product X contain Ingredient Y?").
Q2: Multi-hop, High-visibility (e.g., "Which products made in Country Z contain Ingredient Y?").
Q3: Single-hop, Low-visibility (e.g., "Who is the primary supplier for Company A's raw material?").
Q4: Multi-hop, Low-visibility (e.g., "Trace the supply chain of specific ingredients across multiple product lines to find shared suppliers").

Ground truth is established via a two-stage human annotation process involving primary source compilation and domain expert review.

4. Experimental Results

Experiments compared Helicase against frontier LLMs (zero-shot) and agentic frameworks (ReAct, Tree-of-Thoughts) augmented with web search.

Performance: Frontier LLMs and existing agent frameworks failed on structural supply chain inference, particularly in Q4 (Multi-hop, Low-visibility). They could not generate the necessary network models or produce quantifiable uncertainty.
Helicase's Achievement: Helicase successfully automated the end-to-end discovery process.
- Q4 Accuracy: Achieved a Graph F1 score of 0.85, substantially outperforming the best baseline (ToT at 0.39).
- Uncertainty Calibration: Helicase was the only system producing calibrated uncertainty estimates. It achieved an Uncertainty Calibration Error (UCE) of 0.25, whereas baselines produced no uncertainty estimates.
Ablation Study: Removing key components significantly degraded performance:
- Removing uncertainty-guided planning dropped Graph F1 by 0.12.
- Replacing multiplicative accumulation with a minimum-based rule inflated UCE to 0.41.
- Replacing the multi-agent pipeline with a single-agent loop collapsed Graph F1 to 0.45, demonstrating that the value lies in the tight coupling of uncertainty-guided search and incremental graph construction, not just post-hoc extraction.

5. Significance and Claims

The paper positions Helicase as a shift from passive, retrieval-augmented systems to proactive, uncertainty-guided, graph-based discovery.

Autonomous Discovery: It demonstrates that agentic LLMs can autonomously answer multi-hop, low-visibility supply chain queries that defeat current state-of-the-art models, achieving high accuracy on the hardest benchmark quadrant.
Trustworthiness: By introducing a three-layer uncertainty framework, the system addresses the "hallucination" and "verification" concerns in agentic supply chain management, providing calibrated confidence scores that allow decision-makers to weigh evidence appropriately.
Architectural Insight: The work identifies uncertainty-guided planning, multiplicative evidence accumulation, and multi-agent specialization as the critical architectural components for reliable structural discovery.
Practical Utility: The system reduces the cycle time for multi-tier supplier mapping from weeks of analyst work to minutes, offering a low-cost decision-aid layer for preliminary evidence gathering and uncertainty prioritization.

The authors conclude that this work establishes a new frontier for autonomous information discovery in supply chain research, providing a structured, uncertainty-annotated substrate for downstream analytics and a common benchmark (SCQA) for future evaluation.