Technical Summary: AURA – A Collaborative Reasoning Framework for Anomaly Diagnostics in Underwater Robotics

Problem Statement

The safe deployment of autonomous underwater vehicles (ROVs) in safety-critical environments is hindered by the inability of traditional fault-tolerant control and model-based Fault Detection & Identification (FDI) systems to handle unforeseen, novel, or compound anomalies. While Digital Twins (DT) effectively detect deviations from nominal behavior, they lack the cognitive reasoning required to hypothesize root causes for unmodeled failures. Conversely, directly deploying Large Language Models (LLMs) as autonomous decision-makers introduces significant risks regarding reliability, transparency, and "hallucinations" in critical domains. The core challenge is to create a diagnostic system that leverages the vast knowledge and reasoning capabilities of LLMs while maintaining the safety, trust, and authority of human operators, particularly in remote or secure underwater environments where cloud connectivity may be unavailable.

Methodology: The AURA Framework

The authors propose AURA (Autonomous Resilience Agent), a collaborative framework that synergizes human expertise with AI-driven analysis through a two-agent architecture and a "Human-in-the-Loop Distillation" process.

1. Architecture and Components

• Digital Twin Baseline: AURA utilizes a physics-based Digital Twin (implemented in the Stonefish simulator) running in parallel with the physical ROV. It compares real-time telemetry against the twin's expected behavior using Mahalanobis Distance to detect statistical deviations. The system employs adaptive estimation to handle environmental drift, ensuring only abrupt, fault-induced deviations trigger the diagnostic pipeline.
• Dual-Agent Design:
  • Agent A (State Anomaly Characterisation): A lightweight, locally hosted LLM (4–12B parameters) that translates raw numerical telemetry residuals into a structured, natural-language problem description. It operates on high-frequency signals and is grounded in a local Vector Database (VDB).
  • Agent B (Diagnostic Reasoning): A more powerful, locally hosted LLM (27B parameters) that engages the human operator in an interactive dialogue. It uses Retrieval-Augmented Generation (RAG) to access external knowledge (e.g., technical manuals) and synthesizes hypotheses based on Agent A's characterization and operator input.
• Human-in-the-Loop Interface: The human operator acts as the final arbiter. The system operates as an advisory tool; the operator validates diagnoses and controls physical actions. This interaction occurs via ROS 2 topics, ensuring seamless integration with existing robotic ecosystems.

2. The Learning Loop (Human-in-the-Loop Distillation)

A critical innovation of AURA is its ability to learn from each diagnostic session without computationally expensive retraining:

1. Diagnosis: When an anomaly occurs, Agent A characterizes the issue. Agent B and the operator collaborate to determine the root cause.
2. Distillation: Once the operator validates a diagnosis, the session (raw data, characterization, dialogue, and final diagnosis) is processed into a concise training tuple: (raw anomaly data → correct problem characterization).
3. Storage: This tuple is embedded and stored in a local Vector Database (VDB).
4. Retrieval: In future encounters with similar anomalies, Agent A retrieves these distilled examples to refine its initial characterization from a generic description to a specific, cause-oriented hypothesis, thereby reducing the cognitive load and dialogue turns required from the human operator.

Key Contributions

The paper outlines four primary contributions:

1. A Collaborative Framework: A system that preserves operator authority while utilizing LLMs for root cause analysis, addressing the trust gap in autonomous robotics.
2. Two-Agent Architecture: A separation of perception (Agent A) and reasoning (Agent B) that improves transparency, controllability, and scalability on edge hardware.
3. Retrieval-Augmented Diagnostic Learning: A mechanism that accumulates validated anomaly-solution pairs to support experience-based reasoning, allowing the system to improve progressively without retraining.
4. Natural Language Interface: A user-friendly diagnostic method that enables non-robotic experts to participate effectively in anomaly diagnosis.

Experimental Results

The framework was evaluated using a BlueROV2 in a water tank and a parallel Stonefish digital twin. The study compared system performance in two phases: First Encounter (empty VDB) and Post-Distillation (VDB primed with five human-validated examples).

• Metrics: Performance was measured by the Characterization Specificity Score (CSS) (1–5 scale) and the number of Dialog Turns required to reach a diagnosis.
• Findings:
  • First Encounter: Agent A produced generic, descriptive characterizations (Average CSS: 2.7), requiring an average of 6.2 dialog turns to reach a conclusion.
  • Post-Distillation: After learning from the initial sessions, Agent A generated specific, cause-oriented characterizations (Average CSS: 4.8). This reduced the required human interaction by 71%, dropping the average dialog turns to 1.8.
  • Generalization: The system demonstrated the ability to retrieve relevant context for new, unseen anomalies within the same classes (e.g., thruster disturbances, vertical/rotational motion anomalies), confirming the efficacy of the few-shot learning approach.

Significance and Claims

The paper positions AURA as a step toward trustworthy, continuously improving human-robot teams. The authors claim that the framework successfully transforms the AI from a static tool into an adaptive partner by distilling human expertise directly into the agent's perceptual model. By keeping the LLMs local and the human in the loop for validation, AURA addresses the safety and reliability concerns associated with fully autonomous LLM decision-making.

The authors modestly frame the current work as a proof-of-concept validated in a controlled laboratory environment. They acknowledge that the study relies on a single expert for evaluation and that the "sim-to-real" gap of the digital twin requires further investigation in open-water deployments. Future work is explicitly directed toward field trials with external engineers, refining hydrodynamic parameters for real-world conditions, and exploring automated pre-mission knowledge generation via simulation. The paper does not claim to solve all anomaly detection problems but establishes a new pattern for integrating human feedback into the AI reasoning loop for underwater robotics.