🤖 machine learning

LongAct: Harnessing Intrinsic Activation Patterns for Long-Context Reinforcement Learning

The paper proposes LongAct, a reinforcement learning strategy that improves long-context reasoning by identifying and selectively updating weights associated with high-magnitude, sparse activation patterns, achieving significant performance gains across diverse benchmarks and algorithms.

Original authors: Bowen Ping, Zijun Chen, Tingfeng Hui, Qize Yu, Chenxuan Li, Junchi Yan, Baobao Chang

Published 2026-04-17

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Bowen Ping, Zijun Chen, Tingfeng Hui, Qize Yu, Chenxuan Li, Junchi Yan, Baobao Chang

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a brilliant but overwhelmed librarian (the AI model) who is trying to find a specific fact inside a library that has grown to the size of a small country (the "long context").

Usually, when we train this librarian to get better at their job, we tell them to study every single book on every single shelf equally. We say, "Read page 1 of Book A, then page 1 of Book B, then page 2 of Book A..." This is like trying to memorize an entire encyclopedia by reading every word at the same speed. It's exhausting, inefficient, and often misses the point.

The paper you shared, LongAct, suggests a smarter way to train this librarian. Here is the breakdown using simple analogies:

1. The Discovery: The "Loud" vs. The "Quiet"

The researchers noticed something fascinating while watching the librarian work. When the model processes a huge amount of text, not all parts of its brain are equally active.

The "Quiet" Neurons: Most of the time, the model's internal calculations are very small and faint. They are like background noise or the dust on the shelves.
The "Loud" Neurons: Occasionally, specific parts of the model "shout" with high energy. These are the high-magnitude activations. The researchers realized these "loud" signals are the ones actually doing the heavy lifting—connecting the dots, finding the answer, and making the logic work.

Analogy: Imagine a crowded concert. Most people are just chatting quietly (low magnitude). But every now and then, a few people stand up and scream the lyrics to the chorus (high magnitude). If you want to know what the song is about, you should focus on the people screaming, not the people chatting.

2. The Problem with Current Training

Current training methods (like standard Reinforcement Learning) are like a teacher who says, "Okay, everyone in the class, raise your hand and change your answer if you got it wrong."

This changes everyone's behavior, even the people who were already whispering the right answer.
It wastes energy and can actually confuse the model by messing up the "quiet" parts that were working fine.

3. The Solution: LongAct (The "Spotlight" Method)

LongAct is a new training strategy that acts like a smart spotlight.

Instead of telling the whole model to change, it asks: "Which specific neurons are currently 'shouting' (high magnitude) to solve this problem?"

Step 1: Identify the Stars. The system scans the model's brain and finds the specific "loud" neurons doing the critical work.
Step 2: The Selective Update. It only allows those specific neurons to learn and adjust their weights.
Step 3: Freeze the Rest. The "quiet" neurons are told, "You're doing great, just keep doing what you're doing." They are frozen and don't get updated.

Analogy: Imagine a sports team. Instead of making the whole team run extra laps because one player missed a goal, the coach says, "Only the striker who missed the shot needs to practice shooting. The goalie and defenders stay fresh." This makes the training much more efficient and effective.

4. Why This Matters (The Results)

The paper tested this on huge datasets (like reading a whole novel in one go).

Better Performance: By focusing only on the "loud" parts, the model got significantly smarter at long tasks (about an 8% improvement on tough tests).
Generalization: It didn't just work for one type of problem; it helped the model get better at math, reading comprehension, and logic puzzles of all kinds.
Stability: When they tried to break the model by messing with the "loud" neurons, the model collapsed (started repeating nonsense). But if they messed with the "quiet" neurons, the model kept working perfectly. This proved that the "loud" neurons are the anchors of the model's intelligence.

Summary

LongAct is like teaching a student to study for a massive exam by highlighting only the most important keywords in the textbook, rather than trying to re-read every single page.

Old Way: "Read everything, change everything." (Inefficient, confusing).
LongAct Way: "Find the important parts, fix those, leave the rest alone." (Efficient, powerful, and smart).

This approach allows AI models to handle massive amounts of information without getting overwhelmed, making them much better at reasoning through complex, long stories or documents.

1. Problem Statement

Reinforcement Learning (RL) has become a primary driver for enhancing the reasoning capabilities of Large Language Models (LLMs), particularly in long-context scenarios (e.g., deep dialogue understanding, complex data analysis). However, existing approaches face significant limitations:

External Focus: Current methods primarily optimize external factors such as reward engineering, data synthesis, or progressive context scaling, treating the model's internal computation as a "black box."
Inefficiency: Standard RL training updates all model parameters uniformly, ignoring the fact that long-context reasoning inherently exhibits a sparse structure.
Underutilized Signals: While prior work suggests that hidden dimensions are not equally important (high-magnitude activations encode critical information), long-context RL paradigms largely overlook these intrinsic latent representations.

The core problem is how to leverage the model's intrinsic activation patterns to guide training more efficiently and effectively for long-context reasoning without relying solely on external data or architectural changes.

2. Methodology: LongAct

The authors propose LongAct, a strategy that shifts from uniform parameter updates to saliency-guided sparse updates. The method is grounded in the hypothesis that high-magnitude activations in query ( $Q$ ) and key ( $K$ ) vectors serve as "structural anchors" for long-context reasoning.

Key Technical Components:

Observation of Sparse Activations:
- The authors observe that within the $Q$ and $K$ vectors of attention layers, specific dimensions consistently exhibit high-magnitude activations (outliers), while the rest remain relatively low.
- These high-magnitude activations are identified as critical for encoding information necessary for long-context reasoning.
Saliency-Guided Sparse Update Mechanism:
- Target: The method focuses on the projection weights ( $W_Q$ and $W_K$ ) in the attention layers.
- Dynamic Masking: Instead of updating all weights, LongAct dynamically identifies the "load-bearing" parameters.
  1. Compute the global magnitude matrix ( $M$ ) by calculating the $\ell_2$ -norm of activations across the sequence dimension.
  2. Perform intra-head selection: For each attention head, select the top- $k$ dimensions with the highest magnitudes (controlled by a sparsity ratio $\lambda$ , e.g., 30%).
  3. Map these selected dimensions back to the corresponding rows in the weight matrix ( $W_Q, W_K$ ).
- Gradient Masking: A binary mask ( $G$ ) is applied during backpropagation. Gradients are only updated for the rows corresponding to high-magnitude activations; all other parameters (including $W_V, W_O$ , and MLP layers) receive standard full updates or are frozen depending on the specific configuration, though the paper emphasizes that only the selected rows of $W_Q/W_K$ are updated sparsely while others remain standard.
- Inference: The mask is discarded during inference, meaning the model operates as a standard dense Transformer with zero additional latency.
Training Framework:
- Cold Start: Supervised Fine-Tuning (SFT) on reasoning trajectories to establish a policy prior.
- RL Phase: Uses DAPO (a stable variant of Group Relative Policy Optimization) with a rule-based reward function (format compliance + answer correctness).
- Objective: Optimize the expected reward while restricting gradient flow to the salient weights identified above.

3. Key Contributions

Novel Insight: The paper establishes a direct link between high-magnitude activations in $Q/K$ vectors and long-context reasoning capabilities, validating that these "outliers" are pivotal for model performance.
LongAct Algorithm: Proposes a training framework that selectively updates weights associated with significant activations, moving away from uniform updates.
Universality: Demonstrates that the method is algorithm-agnostic, improving performance across diverse RL algorithms (GRPO, DAPO, KL-Cov).
Efficiency: Achieves performance gains without architectural modifications or inference latency overhead.

4. Experimental Results

The authors evaluated LongAct using Qwen3-8B and Qwen3-4B models across multiple benchmarks.

LongBench v2 (Main Benchmark):
- Performance: LongAct achieved a score of 36.73 on Qwen3-8B, outperforming the standard SFT+DAPO baseline (32.80) by +3.93 and the official Qwen3-8B* release (33.60) by +3.13.
- Hard Tasks: Significant improvements were seen on "Hard" difficulty subsets (+7.63 over DAPO), indicating better handling of error accumulation in complex reasoning.
- Length Robustness: Consistent gains were observed across Short, Medium, and Long context lengths, with the most substantial improvements in the "Long" (>128k) category.
RULER Benchmark:
- LongAct improved the average score on RULER-128K to 51.15 (vs. 49.63 for DAPO), demonstrating enhanced effective context window utilization.
InfiniteBench:
- Achieved the best average performance (49.39 for 8B) across heterogeneous tasks including retrieval, summarization, and QA.
Ablation Studies:
- Activation Selection: Updating weights based on high-magnitude activations yielded the best results (36.73). Random selection or updating low-magnitude weights resulted in significantly lower scores (~29.82).
- Sparsity Ratio: A sparsity ratio of 30% (updating top 30% of weights) was found to be optimal. Higher ratios (40%) led to diminishing returns or regressions.
- Generalization: LongAct also improved performance on short-context benchmarks (GSM8K, HumanEval), proving it is a general robust optimizer, not just a long-context trick.
Perturbation Analysis:
- Disrupting high-magnitude activations (clamping them to the mean) caused immediate model collapse (repetitive loops).
- Disrupting low-magnitude activations preserved reasoning coherence, confirming the critical nature of the high-magnitude "anchors."

5. Significance

Paradigm Shift: LongAct moves the focus of RL optimization from external data/reward engineering to internal model dynamics, offering a new perspective on how to train LLMs for long contexts.
Efficiency & Scalability: By updating only a sparse subset of parameters, the method reduces computational waste during training while maintaining full inference speed.
Interpretability: The work provides empirical evidence that specific activation patterns are structurally responsible for long-context reasoning, bridging the gap between model quantization insights (where high-magnitude values are preserved) and RL training.
Practical Impact: The method is compatible with existing RL algorithms and model architectures, making it a plug-and-play enhancement for improving long-context capabilities in current LLMs.