Technical Summary: Persistent Sparse Autoencoders

1. Problem Statement

Standard Sparse Autoencoders (SAEs) have become a leading tool for interpreting Large Language Models (LLMs) by decomposing dense hidden states into sparse, human-interpretable features. However, standard SAEs operate under an atemporal formulation: they encode each token's hidden state independently. While LLM hidden states inherently integrate preceding context, standard SAEs fail to explicitly represent information that persists across a sequence.

This limitation obscures the "semantic state" of the model—the topic-level context, style, or task state maintained across tokens. Existing temporal extensions of SAEs attempt to address this by pre-assigning specific feature subsets to carry temporal information or using fixed partitions. These approaches lack evidence that such temporal roles can emerge directly from the data and reconstruction objectives. Furthermore, it remains unclear whether standard SAE features implicitly inherit temporal persistence from the hidden states they reconstruct, and if so, how to explicitly model it.

2. Methodology: Persistent Sparse Autoencoders

The authors introduce Persistent Sparse Autoencoders (Persistent SAEs), a framework that extends standard SAEs by learning a persistence coefficient ($\lambda_j$) for each feature $j$.

Core Mechanism

The model separates sparse evidence (token-specific activation) from a persistent state:

1. Evidence Generation: An encoder produces token-specific non-negative sparse activations $a_t \in \mathbb{R}^m_{\ge 0}$, representing new evidence for each feature at token $t$.
2. State Integration: Each feature maintains a decaying state variable $h_t \in \mathbb{R}^m_{\ge 0}$. The state is updated recursively:
   $$h^{(j)}_t = \lambda_j h^{(j)}_{t-1} + (1 - \lambda_j) a^{(j)}_t$$
   • $\lambda_j$ (Persistence Coefficient): Learned during reconstruction.
     • Low $\lambda_j$ (Fast Features): The state decays rapidly, behaving like standard token-specific detectors.
     • High $\lambda_j$ (Slow Features): The state retains evidence over many tokens, integrating sparse updates to track semantic context.
   • The factor $(1-\lambda_j)$ prevents unbounded accumulation, keeping state magnitudes stable.
3. Reconstruction: The decoder reconstructs the hidden state $x_t$ solely from the persistent state $h_t$:
   $$\hat{x}_t = W_{dec} h_t + b_{dec}$$
   The recurrence is diagonal, ensuring feature-level attribution is preserved.

Training Objective

The model is trained to minimize the reconstruction error between the LLM's hidden state and the SAE's output. The objective naturally incentivizes features to learn appropriate timescales: if carrying a feature's state forward improves the reconstruction of future tokens, the optimization favors a higher $\lambda_j$; otherwise, it favors a lower $\lambda_j$. No predefined feature partitions or external temporal objectives are used.

3. Key Contributions

1. Novel Architecture: Introduction of Persistent SAEs, which learn feature-specific persistence coefficients solely through reconstruction, separating local evidence from persistent semantic states without predefined partitions.
2. Empirical Validation of Persistence: Demonstration that standard SAE features implicitly inherit temporal structure from hidden states (signals persist longer in natural documents than in shuffled controls), motivating the explicit modeling of timescales.
3. Disentanglement of Timescales: Showing that Persistent SAEs naturally organize features into a spectrum:
   • Fast features: Locally interpretable detectors for token-specific details.
   • Slow features: A compact subset that integrates evidence to track topic-level semantic states.
4. Application to Monitoring: A case study demonstrating that slow features preserve detection signals for prompt injections over long contexts where token-specific features fail, and that these features remain causally effective in steering model judgments.

4. Experimental Results

Reconstruction and Feature Properties

• Competitive Quality: Persistent SAEs achieve reconstruction quality (FVE and Cosine Similarity) comparable to strong token-specific baselines (BatchTopK, Matryoshka) on Pythia-160M and Gemma-2-2B.
• Timescale Distribution: Learned $\lambda_j$ values span a broad range. Most features are fast (low $\lambda$), with a small tail of highly persistent (high $\lambda$) features.
• Interpretability:
  • Fast features remain locally interpretable, responding to compact lexical or orthographic patterns (e.g., specific substrings).
  • Slow features are harder to interpret in isolation but form coherent clusters when viewed jointly.
• Semantic Clustering: In MMLU subject probing tasks, slow features (top 1% by $\lambda$) form distinct subject-specific clusters in t-SNE visualizations, outperforming raw hidden states and baseline SAEs.
• Unsupervised Selection: Ranking features by decreasing $\lambda_j$ allows for the selection of a compact semantic state that achieves subject-probing AUC nearly as high as a supervised oracle, without using subject labels.

Case Study: Prompt Injection Monitoring

The authors evaluated the ability of features to detect and retain signals of indirect prompt injections (malicious instructions embedded in benign text) over long contexts (up to 350 tokens post-injection).

• Local Detection: Fast features and standard SAEs effectively detect injections at the moment they appear (high accuracy at distance $d=0$).
• Long-Context Retention: As the context extends beyond the injection span, fast features lose predictive power. In contrast, slow features retain the injection signal, maintaining high accuracy at $d=350$.
• Causal Effectiveness: Intervention experiments showed that steering the model using slow-feature directions consistently shifts the monitor's judgment toward flagging the injection, even long after the injection text has passed. Fast features showed near-zero or negative causal effects at these distances.

5. Significance and Claims

The paper claims that Persistent SAEs offer a new mechanism for interpreting and monitoring LLMs by explicitly modeling feature timescales.

• Semantic State Representation: The work suggests that LLMs maintain a "compact semantic state" across tokens, which can be extracted and tracked by slow SAE features without supervision.
• Dual-Role Features: It establishes a functional separation where fast features act as local detectors and slow features act as persistent semantic trackers.
• Monitoring Utility: The ability of slow features to preserve causal signals over long contexts suggests they are a promising tool for safety monitoring, particularly for detecting delayed effects of prompt injections or steering model behavior in long-horizon tasks.

Limitations Acknowledged:
The authors note that the current diagonal recurrence assumes a fixed, single persistence coefficient per feature and cannot model feature interactions or explicit resets. Additionally, the prompt-injection study uses an external monitor; while interventions affect the monitor's judgment, the paper does not claim to have established that these interventions affect the actions of the original agent model.