Neural Associative Skill Memories for safer robotics and modelling human sensorimotor repertoires

Technical Summary: Neural Associative Skill Memories for Safer Robotics

Problem Statement
Modern robotics faces a challenge shared by biological systems: the ability to learn, store, and adaptively express multiple sensorimotor skills while maintaining safety through fault detection. Traditional approaches, such as Associative Skill Memories (ASMs), link movement primitives (e.g., Dynamic Movement Primitives or DMPs) with stereotypical sensory feedback to enable reactive control and anomaly detection. However, existing ASMs rely on hard-coded, dictionary-like libraries where each skill is modeled by a separate set of weights. This architecture necessitates explicit skill selection or manual comparison of sensory profiles to identify the active skill, limiting the system's ability to detect faults across a repertoire without prior instruction. Furthermore, these systems struggle to unify fault detection, reactive control, and skill expression within a single, biologically plausible learning framework.

Methodology
The authors propose Neural Associative Skill Memories (Neural ASMs), a framework that replaces hard-coded libraries with a single, unified neural network based on temporal predictive coding (tPC). This approach utilizes self-supervised learning and biologically plausible local learning rules (Hebbian updates) rather than backpropagation through time (BPTT).

• Architecture: The model employs a two-layer tPC network structured as a Hidden Markov Model (HMM). It consists of hidden states ($z$) representing the inferred context and dynamics, and observation layers ($x$) representing sensorimotor data (e.g., joint angles, gripper forces, visual cues).
• Learning Mechanism: The network learns by minimizing an energy function that represents the prediction error between the current observation and the prediction generated from the hidden state.
  • Memorization: During the learning phase, the network infers hidden states to minimize the sum of squared errors between predicted and actual sensorimotor sequences. Weight updates ($W_H$ for state transitions, $W_F$ for observation prediction) follow local gradient descent rules, creating attractor states in the energy landscape for memorized sequences.
  • Recall (Expression): The system retrieves skills from partial cues via "cued inference." The hidden state is inferred iteratively from initial observations (mimicking motor preparation), setting the initial condition for the dynamical system. The network then predicts future sensorimotor observations. The paper focuses on offline recall, where the trajectory is fixed after the initial cue, modeling ballistic movements.
• Control Integration: Neural ASMs function as a high-level dynamical policy. They do not prescribe a specific low-level controller but guide a flexible control system (e.g., a position controller) by minimizing proprioceptive prediction errors to achieve desired states.

Key Contributions

1. Unified Architecture: The paper demonstrates that a single predictive coding network can learn a repertoire of multiple skills, perform implicit skill recognition, and express them without explicit skill selection mechanisms.
2. Biologically Plausible Learning: Unlike traditional ASMs or standard Recurrent Neural Networks (RNNs) trained via BPTT, Neural ASMs utilize local learning rules and self-supervised predictive coding, offering a more biologically realistic model of sensorimotor learning.
3. Implicit Fault Detection: The framework enables fault detection across all learned behaviors by monitoring the network's energy (prediction error). Abnormalities (out-of-distribution events) result in high energy states, allowing the robot to detect faults without knowing which specific skill is being executed.
4. Contextual Inference and Motor Preparation: The model introduces an iterative inference phase (prospective configuration) that acts as a mechanistic account of motor preparation. This phase allows the network to separate and express distinct skill memories based on contextual cues (e.g., visual targets) prior to movement execution.

Results
The authors validated Neural ASMs through simulation experiments:

• Fault Detection and Isolation: In a pick-and-place task with two skills, the model successfully detected joint-locking faults by identifying abnormal energy levels. It outperformed a baseline method (using Z-score normalized errors) in both fault detection accuracy (82–83% vs. 74–82%) and fault isolation accuracy (79.5% vs. 41%).
• Reactive Correction: The system demonstrated reactive fault correction by minimizing proprioceptive prediction errors in configuration space, allowing a low-level controller to compensate for disturbances (e.g., a collision with a falling object) in real-time.
• Skill Separation and Contextual Inference: Inspired by human motor studies (Sheahan et al., 2016), the model was tested on separating skill memories for counter-clockwise vs. clockwise perturbations under different planning conditions. Neural ASMs successfully separated and expressed the correct motor plans only when contextual cues were available during the planning phase (Planning-only or Planning-and-Execution conditions), replicating human-like adaptation results.
• Speed-Accuracy Trade-off: The model predicted a speed-accuracy trade-off during skill expression: fewer inference iterations (shorter motor preparation time) led to less accurate recall, while increased iterations improved accuracy. This phenomenon was not captured by baseline RNNs trained with BPTT.

Significance and Claims
The paper claims to advance the field of neurorobotics by demonstrating how predictive coding principles can unify fault detection, reactive control, and skill memorization into a single energy-based architecture. The authors posit that Neural ASMs provide a computational lens to study biological sensorimotor learning, specifically:

• Offering a mechanism for injury/fault detection that relies on hierarchical prediction errors rather than peripheral nociceptive signals alone.
• Modeling the role of motor preparatory activity in setting the initial state of a dynamical system to separate skill memories.
• Predicting a speed-accuracy trade-off in habitual skill expression consistent with human experimental data.

The authors maintain a modest stance, acknowledging that their current implementation is a proof-of-concept using simplified simulations (offline recall, limited skill repertoire, and synthetic perturbations). They note that while the framework supports multiple skills, learning rates may depend on the number of skills, and the current model does not yet estimate posterior uncertainty or handle complex sensor faults without dynamic precision adaptation. Future work is suggested to explore online recall, more energy-efficient controllers, and the integration of optimal feedback control.