Factored Reasoning with Inner Speech and Persistent Memory for Evidence-Grounded Human-Robot Interaction

Technical Summary: Factored Reasoning with Inner Speech and Persistent Memory for Evidence-Grounded Human–Robot Interaction

Problem Statement
The transition of assistive robots from controlled demonstrations to long-term, real-world deployment (e.g., in healthcare or domestic settings) necessitates systems that can maintain consistent behavior over days or months, not just single sessions. Current Large Language Model (LLM)-based cognitive architectures often struggle with three critical constraints in long-horizon interactions: (i) detecting when essential information is missing, (ii) distinguishing retrieved evidence from unsupported inferences, and (iii) deciding when to request clarification rather than proceeding with underspecified plans. A significant challenge is the "expressiveness vs. control" tension; while LLMs offer powerful reasoning, they often collapse deliberation, tool invocation, and explanation into a single generation step. This coupling makes intermediate decisions difficult to verify, risks accumulating unverified assumptions into user models, and can lead to "confident but incorrect" responses that undermine trust and safety.

Methodology: The JANUS Architecture
The paper introduces JANUS, a cognitive architecture designed to address these challenges by modeling human-robot interaction as a Partially Observable Markov Decision Process (POMDP). The core innovation is a factored controller that decomposes the interaction into specialized, typed modules with explicit interfaces, separating internal control logic from external communication.

Key architectural components include:

1. Factored Reasoning Pipeline: The system decomposes the input-output mapping into a computational graph of agents:

   • Scope Detection: Identifies the active interaction domain.
   • Intent Recognition & Post-processing: Maps free-form requests to structured task schemas with typed parameters.
   • Memory Agent: A stateful component managing a three-tier memory structure: a bounded recent-history buffer ($H_t$), a compact core memory ($C_t$), and an archival store ($A_t$). It employs an information-sufficiency gate to decide whether archival retrieval is necessary, ensuring the working context ($W_t$) remains bounded and coherent.
   • Inner Speech: An internal control channel that validates intent, checks parameter completeness, and enforces execution readiness. It decides whether the robot should proceed, ask for clarification, switch domains, or reject a request. Crucially, this channel is distinct from the output to the user.
   • Query Generation: Produces executable tool calls (e.g., database queries) only when external evidence is required.
   • Outer Speech: Generates human-facing responses conditioned on the evidence bundle ($B_t = W_t \cup E_t$), ensuring that claims are grounded in retrieved data or explicitly marked as safe defaults.

2. Inner Speech and Control: Inspired by cognitive theories, "Inner Speech" in JANUS acts as a control-oriented internal textual flow. It validates that all required parameters for a task schema are present before allowing grounding. If information is missing, it triggers a clarification turn via Outer Speech rather than hallucinating defaults.

3. Memory Management: Memory is treated as a managed resource rather than an unbounded log. The system uses controlled consolidation and revision policies. When the historic buffer exceeds capacity, traces are summarized and merged into core memory; when core memory exceeds its budget, lower-priority facts are transferred to the archival store. This ensures persistence is bounded, coherent, and auditable.

4. Customization Layer: To support multi-domain operation, JANUS includes a runtime customization layer that encapsulates domain-specific schemas, tools, and prompts without altering the core reasoning logic. This allows the system to switch between domains (e.g., dietary advice, travel planning) dynamically.

Key Contributions
The paper claims three primary contributions:

1. A Modular, Memory-Augmented Factored Architecture: JANUS explicitly separates inner speech (control/verification) from outer speech (communication). This separation allows internal deliberation to be optimized for verification and readiness, while external communication is optimized for clarity and trust calibration.
2. A Dialogue Management Interface for Multi-Domain Switching: The architecture enables run-time switching between domains by encapsulating domain-specific schemes and tools, keeping the core reasoning modules invariant.
3. A Dedicated Memory Agent: A stateful component with hierarchical consolidation capabilities that treats persistence as a managed module with explicit update policies, preventing the accumulation of unstructured history.

Experimental Evaluation and Results
The authors evaluated JANUS using module-level unit tests in a dietary assistance domain (the "Advisor" domain), grounded on a knowledge graph (Neo4j). The evaluation focused on the correctness of intermediate variables rather than just end-to-end response quality.

• Quantitative Results:
  • Scope Detection: Achieved 98.50% accuracy across 11 domains plus a fallback, maintaining high performance even with semantically overlapping domains.
  • Intent Recognition: Achieved 100.00% task-selection accuracy and an F1 score of 1.00 for parameter extraction.
  • Inner Speech: Attained 100.00% accuracy in determining execution readiness (i.e., correctly identifying when clarification is needed).
  • Query Generation: Generated Cypher queries with 100.00% validity (all executed successfully) and a 96.70% overlap score with curated reference results.
• Latency and Efficiency: Median turn-level response times ranged from 0.26s (Intent Recognition) to 0.81s (Inner Speech). The system noted that latency is scenario-dependent, with evidence-backed answers taking approximately 2.46s. Query Generation was the most token-intensive module.
• Qualitative Evaluation: A mixed protocol (human ratings and G-Eval automatic judging) assessed the textual outputs. Results confirmed the architectural hypothesis: Outer Speech was rated highest for clarity and helpfulness, while Inner Speech and Scope Detection (control traces) were rated lower when judged as standalone text. This supports the claim that internal control text is optimized for decision-making, not human readability. Automatic judging (G-Eval) was found to systematically overestimate the quality of control-oriented traces compared to human raters.

Significance and Claims
The paper posits that factored reasoning is a promising path toward scalable, auditable, and evidence-grounded robot assistance. By explicitly separating control decisions from communication, JANUS aims to:

• Prevent the accumulation of unverified assumptions in long-term user models.
• Ensure that robot claims are tied to an evidence bundle, enhancing faithfulness and trust calibration.
• Provide a verifiable control loop where intermediate decisions (domain selection, intent, readiness) can be inspected and measured.

The authors remain modest regarding the current scope, noting that the evaluation relies on curated unit tests and a single domain. They acknowledge that the results do not yet capture multi-turn feedback loops, recovery from upstream errors, or the dynamics of trust in embodied, real-world deployments. Future work is identified as necessary to evaluate these factors through longitudinal, end-to-end studies with real users.