💬 NLP

DAVIS: Planning Agent with Knowledge Graph-Powered Inner Monologue

The paper introduces DAVIS, a novel scientific planning agent that leverages a knowledge graph-powered inner monologue and structured temporal memory to achieve superior performance in complex laboratory tasks and multi-hop question answering compared to existing retrieval-augmented generation approaches.

Original authors: Minh Pham Dinh, Munira Syed, Michael G Yankoski, Trenton W. Ford

Published 2026-03-20

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Minh Pham Dinh, Munira Syed, Michael G Yankoski, Trenton W. Ford

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to teach a robot to perform complex science experiments, like growing a specific type of plant or mixing chemicals. If you just tell the robot, "Go do it," it might panic, forget what it's doing, or accidentally blow up the lab because it doesn't understand the sequence of events or the consequences of its actions.

This paper introduces DAVIS, a new kind of AI agent designed to be a careful, thoughtful scientist assistant. Here is how it works, explained through simple analogies:

1. The Problem: The "Amnesiac" Robot

Most current AI agents are like amnesiac tourists. They can read a map (the internet) and answer questions, but they don't have a good memory of what happened five minutes ago or how the world changes over time.

The Issue: If a robot opens a cupboard and finds a kettle, a standard AI might forget that the kettle is there once it turns around. It struggles with tasks that require a chain of events (e.g., "You must boil water before you can make tea").
The Result: They often fail at science tasks because they lack "common sense" about time and cause-and-effect.

2. The Solution: DAVIS's "Inner Monologue"

DAVIS is different because it talks to itself. The authors call this an "Inner Monologue."

Think of DAVIS not as a robot that just reacts, but as a detective solving a mystery.

The Detective's Notebook (The World Model): Instead of just looking at the room, DAVIS keeps a detailed, living notebook called a Temporal Knowledge Graph. This isn't just a list of facts; it's a timeline. It records: "At 10:00 AM, I picked up the beaker. At 10:05 AM, I poured water into it. At 10:10 AM, the water started boiling."
The Inner Dialogue: Before making a move, DAVIS asks itself questions like a human thinking out loud: "Wait, I need hot water. Did I already heat the water? If I pour cold water now, will it break the glass? Let me check my notebook."

3. How It Works: The "Actor-Critic" Team

DAVIS uses a three-person team inside its brain to make decisions:

The Planner (The World Model): This is the strategist. It looks at the "notebook" (the knowledge graph) and predicts the future. It says, "If I do X, then Y will happen." It creates a step-by-step plan.
The Actor (The Doer): This is the hands. It takes the high-level plan from the Planner and breaks it down into tiny, physical commands like "pick up," "turn on," or "pour."
The Critic (The Safety Inspector): This is the most important part. After the Actor does something, the Critic checks the result.
- Scenario: The plan was "Heat the water." The Actor turns on the stove.
- The Check: The Critic looks at the result. Is the water hot? If the stove was broken and the water didn't heat up, the Critic says, "Stop! That didn't work. Let's rethink the plan."
- This prevents the robot from blindly repeating mistakes.

4. The "Science World" Test

The researchers tested DAVIS in a video game called ScienceWorld, which simulates a lab with 9 different subjects (like biology, chemistry, and physics).

The Competition: They pitted DAVIS against other smart AI agents (like ReAct, SayCan, and Reflexion).
The Result: DAVIS won in 8 out of 9 subjects. It was much better at figuring out the right order of steps and didn't get confused when things went wrong. It was like a student who actually studied the textbook versus a student who just guessed.

5. Why This Matters

Safety: Because DAVIS "thinks" before it acts and checks its work, it's much safer. It won't mix dangerous chemicals unless it's sure of the steps.
Reasoning: It can connect dots. If it learns that "fire needs oxygen," it can apply that to a new situation it's never seen before.
Transparency: You can read its "Inner Monologue" to see exactly why it made a decision. This is huge for science, where you need to know how a result was achieved, not just that it happened.

The Catch (Limitations)

The paper admits DAVIS is currently expensive and slow.

The Cost: Because it talks to itself so much and checks its "notebook" constantly, it uses a lot of computer power (like a supercomputer thinking hard). Doing one experiment might cost a few dollars in computing fees.
The Dependency: It relies heavily on the "brain" (the Large Language Model) it uses. If that brain changes or makes a mistake, DAVIS gets confused.

Summary

DAVIS is like giving a robot a memory, a conscience, and a voice. Instead of just reacting to the world, it pauses, checks its notes, talks to itself to fill in the gaps, and only then takes action. This makes it a much better candidate for helping humans in delicate, complex environments like scientific laboratories.

1. Problem Statement

The paper addresses the challenge of designing generalist scientific agents capable of autonomously performing complex tasks in laboratory settings. Unlike everyday tasks, scientific tasks require:

High-level reasoning: Multi-hop reasoning over domain-specific procedural knowledge.
Temporal and structured understanding: Modeling how environments evolve over time, including delayed or compounding effects of actions.
Safety and interpretability: Justifying actions and ensuring physical consequences are safe.
Partial observability: Operating with limited real-time feedback.

Existing approaches, such as standard Reinforcement Learning (RL) or Retrieval-Augmented Generation (RAG) with unstructured vector databases, often fail to meet these requirements. They suffer from sample inefficiency, lack of long-term memory, inability to perform causal/multi-hop reasoning, and static retrieval methods that prevent dynamic adaptation.

2. Methodology: The DAVIS Framework

DAVIS (Davis Institute for Artificial Intelligence) is a model-based planning agent that integrates a Temporal Knowledge Graph (TKG) with an agentic, multi-turn retrieval system (termed "Inner Monologue").

Core Components

World Model (WM) as a Temporal Knowledge Graph:
- Instead of static vector storage, DAVIS constructs a TKG ( $G_t = (E, R, T)$ ) where entities, relations, and timestamps evolve as the agent interacts with the environment.
- Construction: Transitions (observation $\parallel$ action $\parallel$ next observation) are summarized by an LLM, resolved for coreference using Stanford CoreNLP, and parsed into entity-relation tuples with timestamps.
- Function: The TKG serves as structured, interpretable external memory, enabling the agent to infer causal relationships and state transitions over time.
Inner Monologue Retrieval System:
- Unlike traditional RAG which retrieves static information, DAVIS engages in an iterative, conversational process with its WM.
- Process: When facing a query (e.g., "Where is the water?"), the agent:
  1. Identifies relevant entity types (e.g., Location, Person).
  2. Iteratively expands the search by adding neighbors to form a maximal subgraph.
  3. Reorders edges based on timestamps to establish a coherent sequence of events.
  4. Passes this temporal sequence to an LLM for extrapolation and summarization to fill knowledge gaps.
Actor-Critic Architecture with Planning:
- Planner (World Model): Generates high-level, natural language action trajectories ( $\tau$ ) based on the approximated belief state and retrieved history.
- Actor ( $R_a$ ): Decomposes high-level plans into executable low-level commands for the environment.
- Critic ( $R_c$ ): Continuously evaluates the execution by comparing predicted outcomes (from the WM) with actual environment responses.
  - If a discrepancy is found (e.g., a tool is broken), the Critic triggers replanning.
  - This reflection loop updates the agent's internal monologue and the TKG, allowing for dynamic adaptation.
Mathematical Formulation:
- The problem is modeled as a Partially Observable Markov Decision Process (POMDP).
- DAVIS approximates the belief state $\hat{b}_t$ and learns transition/reward models ( $\hat{T}, \hat{R}$ ) using the TKG and the inner monologue $M_t$ .
- The policy is defined as $\pi(a_t | \hat{b}_t, M_t)$ , optimizing for cumulative discounted rewards.

3. Key Contributions

Agentic Multi-Turn Retrieval: DAVIS is the first RAG agent to employ an interactive retrieval method where the agent iteratively queries its memory (Inner Monologue) to refine understanding before acting, rather than relying on one-shot static retrieval.
Temporal Knowledge Graph Memory: Introduces a structured memory system that explicitly models time and causality, enabling multi-hop reasoning and handling delayed effects in scientific tasks.
Actor-Critic with Pre-Execution Deliberation: Unlike frameworks that evaluate only after task completion, DAVIS uses a Critic to validate steps in real-time and trigger replanning, bridging high-level planning with low-level execution.
Interpretability: The use of natural language plans and a structured graph allows human supervisors to inspect the agent's reasoning process, addressing safety concerns common in "black-box" RL agents.

4. Experimental Results

The authors evaluated DAVIS on the ScienceWorld benchmark (9 grade-school science subjects) and multi-hop QA datasets.

ScienceWorld Performance:
- DAVIS outperformed state-of-the-art baselines (SayCan, ReAct, Reflexion, RAP) on 8 out of 9 science subjects.
- It achieved an overall average score of 65.06, approximately 1.8 times higher than competing methods.
- It required fewer steps to converge and demonstrated superior knowledge transfer across task variations.
Ablation Studies:
- Without World Model (WM): Performance dropped significantly (e.g., from 70.00 to 3.00 on "Melt" tasks), proving that structured temporal memory is critical for complex reasoning.
- Without Critic: The agent struggled to recover from errors in long-horizon tasks, highlighting the importance of real-time reflection and replanning.
- Without Actor: The agent failed to decompose goals into executable commands, leading to incoherent action sequences.
Multi-Hop QA (HotpotQA & MusiqueQA):
- DAVIS's WM achieved competitive results, surpassing GraphReader and GraphRAG on HotpotQA (F1: 73.8) and performing strongly on MusiqueQA.
- This demonstrates the framework's ability to generalize to general knowledge reasoning tasks, not just scientific simulations.

5. Significance and Limitations

Significance:

Scientific AI: DAVIS represents a major step toward autonomous scientific assistants that can safely navigate complex, procedural environments.
Reasoning Paradigm: It shifts the paradigm from static retrieval to dynamic, iterative reasoning, mimicking human "inner monologue" to fill knowledge gaps.
Safety: By validating plans against a World Model before execution, it offers a safer alternative to trial-and-error RL in critical domains.

Limitations & Future Work:

Computational Cost: The reliance on LLMs for every step of the inner monologue results in high token usage (~~43k tokens/action) and cost (~~$0.43/action). Future work aims to distill this into smaller, open-source models.
LLM Sensitivity: Performance is tightly coupled with the underlying LLM's stability and quality; changes in API or model weights can disrupt the agent.
Unimodal: Currently operates only in text-based environments. Future iterations need to integrate visual and sensory inputs for real-world robotic deployment.
Bias: The agent's planning is dependent on the TKG; if the graph lacks specific knowledge, the agent may struggle with novel scenarios.

In conclusion, DAVIS establishes a new benchmark for agentic reasoning in scientific domains by combining structured temporal memory with iterative self-reflection, offering a robust, interpretable, and highly effective framework for complex task planning.