Free(): Learning to Forget in Malloc-Only Reasoning Models

Technical Summary: Free()LM: Learning to Forget in Malloc-Only Reasoning Models

1. The Problem: The "Malloc-Only" Paradox

Current reasoning models (such as the Qwen3 or DeepSeek series) scale performance by increasing "test-time compute"—generating more thinking tokens to solve complex problems. However, the authors identify a fundamental architectural flaw: these models operate as "malloc-only" engines. They continuously accumulate every reasoning step, intermediate thought, and trial path into their context window without any mechanism to discard obsolete or redundant information.

This leads to a critical bottleneck:

• Context Pollution: As reasoning trajectories grow (especially beyond 16k–80k tokens), the accumulation of "noise" (repetitive loops, dead-end logic, and redundant steps) degrades the model's ability to focus on the actual solution.
• Catastrophic Collapse: In long-horizon tasks, standard models eventually suffer from total reasoning collapse (dropping to 0% accuracy) because the context window becomes overwhelmed by its own "thoughts."

2. Methodology: The Free()LM Framework

To solve this, the authors propose Free()LM, a model that introduces an intrinsic self-forgetting capability. Instead of just "allocating" memory (adding tokens), the model learns to "free" it (prune tokens).

A. Architecture: The Free-Module
The core innovation is a plug-and-play LoRA adapter called the Free-Module. The model operates in a cyclic two-mode inference process:

1. Reasoning Mode (Unmerged): The LoRA is unmerged; the model behaves like the original backbone, focusing purely on generating the next reasoning tokens.
2. Cleaning Mode (Merged): Triggered every $L_{clean}$ tokens, the LoRA is merged into the backbone. The model stops reasoning and scans the context to identify redundant segments. It outputs a structured JSON pruning command containing a prefix and suffix that act as anchors to define the span of text to be deleted.

B. Efficient Pruning & Resumption
Once the command is issued, an external executor prunes the text. To resume reasoning without losing coherence, the authors use a Re-prefilling strategy: they reuse the KV cache for the unchanged prefix and strictly re-prefill the altered suffix, ensuring compatibility with standard serving frameworks like vLLM.

C. Training: Learning to Forget via Reward-Based Synthesis
Since there are no ground-truth labels for "what to forget," the authors developed a sophisticated data pipeline:

• Data Synthesis: They use a strong teacher model (Gemini-2.5-Pro) to generate candidate pruning commands on reasoning trajectories.
• Reward Mechanism (Rejection Sampling): To ensure the model doesn't prune essential information, they only keep training instances where the pruned context ($C_{new}$) maintains or improves accuracy compared to the original context ($C_{raw}$). This is verified by running multiple parallel reasoning rollouts.

3. Key Contributions

• New Paradigm: Shifts LLM reasoning from a "malloc-only" (append-only) approach to a "malloc + free" (active memory management) approach.
• Plug-and-Play Design: The Free-Module is a lightweight LoRA, making it easy to integrate into existing large-scale models.
• Universal Capability: The authors demonstrate that a Free-Module trained on a small model (8B) can generalize and improve the performance of much larger or even "alien" architectures (e.g., DeepSeek-V3.2).

4. Results

The experiments across models ranging from 8B to 685B parameters yielded impressive results:

• Performance Boost: Free()LM achieved a 3.3% average improvement across six reasoning benchmarks and set a new SOTA on IMOanswerBench using DeepSeek V3.2-Speciale.
• Solving Collapse: In long-horizon tasks where the standard Qwen3-235B model dropped to 0% accuracy, Free()LM restored performance to ~50%.
• Efficiency: While it introduces a latency overhead (due to the cleaning cycles), it significantly reduces memory pressure, achieving a ~45% reduction in KV cache usage.
• Precision: Unlike heuristic methods (like H2O or ThinkClearly) which often disrupt reasoning and cause loops, Free()LM's learned pruning is precise, avoiding the "prune-then-regenerate" cycle seen in ICL-based methods.

5. Significance

The paper concludes that sustainable intelligence requires the freedom to forget. As reasoning models move toward increasingly long-horizon tasks (100k+ tokens), simply expanding the context window is insufficient. The ability to actively manage and prune the internal "workspace" is essential for preventing cognitive overload and ensuring that scaling test-time compute leads to actual intelligence rather than just increased noise.