💻 computer science

Cognitively-Grounded On-Device Runtime Learning for Ground Robots in Unknown Physical Environments

This paper introduces CogRun, a fully autonomous on-device framework that enables safety-critical ground robots to perform cognitively-grounded runtime learning in unknown environments without prior maps or connectivity by integrating a novel learning agent with a dedicated safety-focused rational agent.

Original authors: Yihao Cai, Yanbing Mao, Christian Lebiere

Published 2026-09-01

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Yihao Cai, Yanbing Mao, Christian Lebiere

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine a robot sent into a forest it has never seen before, tasked with navigating a path littered with fallen branches, hidden holes, and uneven ground. In the world of artificial intelligence, this scenario represents a fundamental hurdle. Most robots today are trained in perfect, simulated worlds or controlled laboratories, learning to move in environments that are predictable and static. When these machines are deployed into the chaotic, shifting reality of a wild forest or a disaster zone, they often struggle. Their pre-programmed knowledge becomes a liability because the world they face does not match the world they were taught. Furthermore, these robots often rely on a constant connection to powerful computers in the cloud to process their decisions. In remote, rugged terrain, that connection is frequently broken, leaving the robot stranded or unable to adapt in real time. The challenge, then, is to create a machine that can learn on the fly, entirely by itself, while remaining safe enough to operate without human intervention.

A team of researchers has addressed this challenge with a new system called CogRun, designed to let ground robots learn safely while they are actually moving through unknown physical spaces. The core idea is to give the robot a form of on-the-spot learning that happens entirely on its own onboard computer, without needing to send data back to a server. This approach allows the robot to adapt to new obstacles and changing conditions the moment it encounters them. The system is built on the understanding that learning in the real world is dangerous; a robot that simply tries new things to see what works might fall over or crash. To solve this, the researchers designed a framework where the robot's learning process is constantly monitored and guided by a separate, non-learning safety system. This ensures that while the robot is figuring out how to move better, it never takes a step that could cause it to tip over or collide with something.

The framework operates by splitting the robot's brain into three distinct parts that work together. The first part is the "Learning Agent," which is the curious explorer. It tries out different movements, observes the results, and tries to improve its performance based on what it experiences. However, because this agent is still learning, it can make mistakes. To prevent those mistakes from becoming disasters, a second part, the "Rational Agent," acts as a strict safety guardian. This agent does not learn; it relies on established, proven rules to ensure the robot stays upright and avoids collisions. If the Learning Agent suggests a move that looks risky, the Rational Agent immediately takes over control to execute a safe maneuver instead. A third component, the "Coordinator," watches the robot's state in real time and decides which agent is in charge at any given moment. If the robot is moving safely, the Coordinator lets the Learning Agent drive. The moment the robot enters a dangerous situation, the Coordinator hands the controls to the Rational Agent until the danger passes.

What makes this system unique is how it handles the memories of its experiences. When a robot learns, it generates a massive amount of data about what it did and what happened next. Most systems store this data in a simple list and pick examples at random to learn from. The researchers found this inefficient, especially in a changing environment where old or irrelevant data can confuse the learning process. Instead, CogRun uses a method inspired by how human memory works. It prioritizes memories based on three factors: how similar a past experience is to the current situation, how recently it happened, and how often similar situations have occurred. This allows the robot to focus its learning on the most relevant moments, such as the exact time it almost slipped on a wet leaf, rather than wasting time on data from hours ago when the conditions were completely different.

To test this system, the researchers deployed it on a four-legged robot in a real, unknown forest. The environment was filled with dead zones, deep depressions in the ground, and holes covered by leaves that could hide a fall. The robot was tasked with traveling thirty meters from a starting point to a goal. In these tests, the robot had to learn to navigate the terrain without any prior map. The results showed that the system successfully kept the robot safe throughout the entire learning process. While a standard safety system could keep the robot from falling, it was often too cautious to move efficiently. The CogRun system, however, allowed the robot to learn a smooth, efficient path in just a few dozen attempts. The robot learned to adjust its gait and speed to handle the uneven ground, eventually reaching its goal much faster than it could have with a fixed, pre-programmed strategy.

The team also tested the system on an off-road autonomous vehicle in a simulated forest environment to see if the approach worked on different types of machines. They compared their system against other advanced learning methods that try to handle safety. In these simulations, the other systems frequently lost their balance or crashed into obstacles, often failing to complete the task. In contrast, the CogRun system completed the vast majority of its runs without any safety violations. The robot learned to navigate through dense trees and rocky paths, maintaining stability even when the terrain changed unexpectedly. The researchers noted that the system's ability to blend the safe, rule-based actions with the learning-based actions was crucial. By carefully mixing the cautious moves of the safety guardian with the adaptive moves of the learning agent, the robot could explore new solutions without ever crossing the line into danger.

This work demonstrates that it is possible for robots to learn continuously in the real world without relying on a connection to the cloud or a pre-existing map of the environment. By combining a learning agent that seeks improvement with a rational agent that guarantees safety, and by using a memory system that prioritizes the most relevant experiences, the researchers have created a framework that allows machines to adapt to the unpredictable nature of the physical world. The experiments suggest that this approach can significantly improve both the safety and the efficiency of robots operating in complex, unknown environments, paving the way for more capable machines that can assist in disaster recovery, exploration, and other critical tasks where the terrain is too dangerous or unpredictable for human operators.

Technical Summary: CogRun – Cognitively-Grounded On-Device Runtime Learning for Ground Robots

1. Problem Statement

The paper addresses the limitations of current AI-embodied robots when operating in unstructured, dynamic, and unknown physical environments (e.g., wild forests, post-disaster zones). It identifies three fundamental challenges:

Non-Stationary and Unknown Environments: Existing "train-then-deploy" paradigms suffer from sim-to-real gaps and domain shifts, leading to performance degradation in environments that cannot be fully characterized prior to deployment.
Communication-Limited Constraints: Current learning frameworks often rely on cloud-edge architectures. This dependence on wireless connectivity is impractical for remote, chaotic environments where communication is unreliable or absent, and the inherent latency precludes real-time safety-critical adaptation.
Inefficient Experience Replay: Standard experience replay strategies often fail to account for the relevance of stored experiences to the robot's current state. In complex, non-stationary environments, retrieving outdated or irrelevant data degrades learning efficiency, while on-policy methods suffer from poor sample efficiency.

2. Methodology: The CogRun Framework

To address these challenges, the authors propose CogRun, a framework for cognitively-grounded, on-device runtime learning. The system operates entirely on edge-AI hardware (specifically NVIDIA Jetson AGX Orin), eliminating cloud dependence. CogRun consists of three tightly coupled components:

A. The Coordinator
The Coordinator acts as the safety gatekeeper. It monitors the robot's real-time state $s(t)$ against a pre-defined Safety Set $S$ (e.g., collision avoidance, velocity limits).

If $s(t) \in S$ , the Coordinator executes the action $a_{learning}(t)$ generated by the Learning-Agent.
If $s(t) \notin S$ , the Coordinator immediately switches control to the Rational-Agent, executing $a_{rational}(t)$ to ensure safety.
Once the state returns to the safety set, control reverts to the Learning-Agent.

B. The Rational-Agent
This is a non-learning module dedicated exclusively to safety-critical functions. It serves as a verifiable safety fallback (e.g., using Interactive-FAR or physics-model-based controllers). When activated, it prevents catastrophic failures and shares its "safety-assurance experiences" with the Learning-Agent to reduce risky trial-and-error exploration.

C. The Learning-Agent
This component features a novel cognitive-neural learning architecture designed for runtime adaptation. It integrates Reinforcement Learning (RL) with Instance-Based Learning (IBL) through three key innovations:

Dedicated Replay Buffers: Two separate buffers are maintained:
- $B_L$ : Stores task-oriented experiences from the Learning-Agent's exploration.
- $B_R$ : Stores safety-assured experiences generated by the Rational-Agent during interventions.
- Mini-batches for training are constructed by sampling from both buffers, allowing the agent to learn safe strategies directly from the Rational-Agent's interventions.
Cognition-Driven Batch Sampling: Inspired by human memory retrieval principles, the system ranks stored experiences based on three metrics relative to the current state:
- Similarity: Structural and surface-level similarity to the current context.
- Frequency: How often a specific type of experience has been encountered.
- Recency: How recently the experience occurred.
- A concurrent frequency-recency measure is computed using a memory decay factor and "family" clustering of experiences.
Safety-Aware RL-IBL Blending: The agent does not rely solely on the neural network's output ( $a_{actor}$ ). Instead, it blends the actor's action with an "experience action" ( $a_{experience}$ ) derived from the retrieved rational instances. A state-dependent blending parameter $\eta(s)$ , formulated via convex optimization to ensure safety constraints, dynamically weights these two actions:
$a_{learning} = (1 - \eta(s)) \cdot a_{actor} + \eta(s) \cdot a_{experience}$

D. Edge-AI Execution
The framework utilizes a decoupled multi-rate execution strategy on the edge device:

CPU (ARM-based): Runs the Coordinator and Rational-Agent at high frequencies (200–1000 Hz) for real-time safety assurance.
GPU: Runs the Learning-Agent at lower frequencies (10–20 Hz) for policy updates.
This ensures temporal consistency and low latency while maintaining energy efficiency.

3. Key Contributions

Full On-Device Autonomy: CogRun provides a complete autonomy stack (perception, learning, control) on edge devices, removing reliance on cloud connectivity for safety-critical runtime learning.
Cognitive-Neural Architecture: The integration of IBL with Actor-Critic RL via dedicated buffers and cognition-driven sampling offers a principled approach to experience efficiency in non-stationary environments.
Safety-Aware Blending: The mechanism of blending RL actions with IBL-derived rational actions allows for continuous adaptation while strictly adhering to safety constraints.

4. Experimental Results

The authors validated CogRun through two primary experimental setups:

A. Quadruped Robot in Real-World Wild Forests

Tasks: Runtime navigation and end-to-end vision-to-action learning in unknown forests with obstacles like dead zones, swales, and leaf-covered holes.
Findings:
- CogRun strictly assured safety (zero collisions or falls) throughout the learning process, whereas baseline fault-tolerant RL frameworks (Real-DRL, RLM, MPS) suffered from balance loss and collisions.
- The robot achieved stable, efficient navigation within tens of episodes, significantly outperforming baselines in task completion rates (19/20 for CogRun vs. 10–11/20 for baselines).
- The system demonstrated robustness by transferring learned policies across three distinct, non-stationary forest environments without reinitialization.

B. Off-Road Autonomous Vehicle (Gazebo Simulation)

Ablation Study: The authors compared CogRun against three configurations: "Random" (standard RL), "Cognition" (cognition-driven sampling only), and "Cognition+Blending" (full CogRun).
Findings:
- "Cognition" consistently outperformed "Random."
- "Cognition+Blending" achieved the highest rewards, confirming that the safety-aware RL-IBL blending mechanism significantly enhances both safety and task performance.

5. Significance and Claims

The paper claims that CogRun represents a significant step toward achieving human-level autonomy in unstructured physical environments. Its primary significance lies in:

Enabling Safe Runtime Adaptation: It allows robots to continuously learn and adapt to unknown conditions without prior maps or perceptual knowledge, while guaranteeing safety through a dual-agent architecture.
Overcoming Connectivity Barriers: By operating entirely on edge-AI devices, it makes runtime learning feasible in remote, communication-limited scenarios where cloud-assisted learning is impossible.
Improving Sample Efficiency: The cognitively-grounded approach to experience replay addresses the inefficiency of standard RL in physical robots, enabling effective learning from scarce real-world interactions.

The authors conclude that while the framework is effective, future work will focus on refining principled experience ranking, memory decay mechanisms, and more effective measures of frequency, recency, and similarity to further advance cognitively grounded runtime learning.