Technical Summary: Neural Procedural Memory (NPM)

Problem Statement

While Large Language Models (LLMs) excel as static solvers, transforming them into autonomous agents capable of operating in dynamic, open-ended environments remains challenging. A critical bottleneck in this transition is the lack of persistent procedural memory. Current agent architectures primarily rely on Retrieval-Augmented Generation (RAG) to inject explicit textual guidelines (e.g., workflows, insights) into the context window.

However, the paper argues that relying solely on symbolic text instructions introduces a "text-action disconnect." Procedural knowledge is often implicit and abstract; compressing complex reasoning patterns into discrete natural language tokens inevitably leads to information loss. Consequently, agents may comprehend retrieved instructions but fail to strictly act upon them, often omitting critical intermediate steps or failing to align retrieved workflows with the execution trajectory. This disconnect is particularly pronounced in long-horizon tasks where maintaining state and adhering to sequential logic is difficult via text alone.

Methodology: Neural Procedural Memory (NPM)

To address these limitations, the authors propose Neural Procedural Memory (NPM), a training-free framework that represents agent memory through implicit activation steering rather than explicit textual instructions. Instead of retrieving text for the agent to read, NPM distills procedural skills from historical contrastive experiences into steering vectors within the model's activation space.

The framework operates in three phases:

1. Contrastive Experience Construction

NPM constructs a dataset of paired reasoning segments $(x^+, x^-)$ by isolating effective reasoning patterns from historical failures. It employs a dual-granularity strategy to capture procedural signals:

• Inter-trajectory Contrast: Aligns successful trajectories ($\tau^+$) with failed attempts ($\tau^-$) for a specific task to capture global behavioral shifts.
• Intra-trajectory Contrast: Exploits step-level differences within a single failed trajectory. It identifies "degenerate steps" (e.g., redundant loops, invalid actions) and contrasts them against "effective steps" within the same trajectory. This allows learning from failure even when successful trajectories are sparse.

2. Procedural Memory Extraction

The framework maps the textual contrasts into continuous geometric transformations in the representation space.

• Representation: For inter-trajectory pairs, it extracts the last token hidden state of the full trajectory. For intra-trajectory pairs, it applies mean-pooling over tokens to obtain step representations.
• Vector Derivation: The system computes the difference between the representations of successful ($h^+$) and degenerate ($h^-$) states. Using Principal Component Analysis (PCA), it extracts the dominant direction of variation to form a steering vector $v$ that represents the shift from failure to success. These vectors are stored in a task-specific repository.

3. Inference-Time Intervention

During inference, NPM dynamically synthesizes and injects steering vectors without modifying model weights or expanding the context window:

1. Retrieval: A dense retriever identifies the top-$K$ most similar historical tasks for the current query.
2. Synthesis: The system fetches stored representations from the retrieved tasks to form a collective memory pool. A task-specific consensus direction $v_l(q)$ is derived via an extraction function.
3. Steering: The synthesized vector is injected into the residual stream at each time step $t$ for target layers $l$:
   $$\tilde{h}_{l,t} = h_{l,t} + \alpha \cdot v_l(q)$$
   where $\alpha$ controls the intervention strength. This directly modulates the agent's reasoning process and action selection.

Key Contributions

1. NPM Framework: The proposal of a training-free framework that integrates implicit activation steering into agent memory, enabling effective intrinsic behavioral modulation without parameter updates.
2. Dual-Granularity Extraction: The introduction of a method using both inter-trajectory and intra-trajectory contrasts. This offers a flexible intervention mechanism that mitigates reliance on successful demonstrations by enabling learning from failure.
3. Empirical Validation: Extensive experiments demonstrating that activation steering serves as an effective carrier for agentic skills, yielding interpretable vectors that correlate with specific task behaviors.

Experimental Results

The authors evaluated NPM across four agent benchmarks: ALFWorld, WebShop, ScienceWorld, and BabyAI, using MiniCPM3-4B and Qwen3 (4B/8B) models.

• Performance vs. Baselines: NPM outperforms static implicit baselines (CAA, Mass-Mean) and achieves performance comparable to explicit textual memory baselines (Insights, Workflows). In several settings (e.g., WebShop with Qwen3-4B), NPM outperformed explicit workflows.
• Hybrid Synergy: Combining implicit steering with explicit workflows (Hybrid configuration) yielded the highest overall performance across all models. This suggests that explicit text provides high-level symbolic planning, while implicit steering reinforces procedural adherence directly within neural representations.
• Efficiency: Unlike textual memory, which increases prefill latency significantly due to context window expansion, NPM maintains latency nearly identical to the "No Memory" baseline, as vector injection is a simple element-wise addition.
• Interpretability: Representational analyses (PCA projections, linear separability tests) confirmed that successful and failed modes occupy distinct regions in the activation space. Feature decomposition revealed that steering vectors encode consistent task logic, such as suppressing "redundant observation" or amplifying "systematic searching."

Significance and Claims

The paper claims that implicit activation steering provides a promising and practical approach for managing procedural memory in LLM agents. By shifting from external verbalization to internal activation modulation, NPM addresses the text-action disconnect inherent in RAG-based procedural memory.

The authors emphasize that their approach offers a complementary advantage to existing methods: while textual memory supplies symbolic guidance, implicit steering ensures adherence to these plans during long execution horizons by directly modulating the generation path. The framework provides a foundation for managing procedural memory without the computational overhead of fine-tuning or the context limitations of text-based retrieval.

Limitations Acknowledged:
The paper notes that NPM requires access to internal activation spaces (restricting application to open architectures), relies on heuristics to identify degenerate steps (which may miss subtle logical errors), and faces a cold-start problem in environments where agents rarely generate successful trajectories initially. Additionally, the current static modulation lacks the flexibility to shift between behavioral primitives across different execution stages dynamically.