Technical Summary: TYLER (Typed Latent Reasoning)

1. Problem Statement

Large Language Models (LLMs) currently rely on Chain-of-Thought (CoT) prompting to solve complex reasoning tasks. While effective, CoT externalizes intermediate computation as discrete text tokens, introducing redundancy, increasing inference costs, and imposing a rigid interface where reasoning must be fully articulated before the final answer.

Latent reasoning offers an alternative by keeping intermediate computation in continuous representations. However, existing latent reasoning methods suffer from two primary limitations:

1. Lack of Functional Differentiation: Current approaches typically treat latent tokens as a uniform, undifferentiated carrier of computation. They fail to distinguish between different functional roles of reasoning steps, such as global orientation, local state updates, or procedural abstraction.
2. Rigid or External Control: Existing methods either predefine the position of latent computation (fixed-position) or rely on external triggers/uncertainty signals to decide when to invoke latent computation. They do not enable the LLM itself to dynamically decide when to think, what type of computation to perform, or how much computational budget to allocate during autoregressive decoding.

2. Methodology: TYLER Framework

The authors propose TYLER (Typed Latent Reasoning), a framework that reframes latent reasoning as an online, typed, and budgeted computation problem. The core innovation is a policy that interleaves visible token decoding with the invocation of specialized latent operators.

2.1 Typed Latent Operators

TYLER introduces three distinct latent operators, each designed for a specific computational role:

• $O_g$ (Global Orientation): Supports global planning and problem orientation.
• $O_s$ (Local State Update): Supports updates to the evolving reasoning state.
• $O_p$ (Procedural Abstraction): Captures reusable procedural patterns.

Instead of a single latent space, these operators map the current reasoning context to sequences of operator-conditioned latent tokens.

2.2 Architecture and Decoding

The framework extends the standard autoregressive decoding interface:

• Operator Invocation: The model's vocabulary is extended with special tokens ($u_g, u_s, u_p$) corresponding to the three operators. At each decoding step, the model chooses between emitting a visible text token or invoking one of the typed operators.
• Latent Synthesis: When an operator is invoked, a Latent Synthesis Module generates a sequence of latent tokens. This module uses LoRA (Low-Rank Adaptation) layers added to the backbone (keeping original parameters frozen) and utilizes operator-specific query tokens and projection heads to synthesize tokens compatible with the LLM's input embedding space.
• Context Integration: The synthesized latent tokens are appended to the context as "silent" computation, guiding subsequent visible decoding without being exposed to the user.

2.3 Two-Stage Training Procedure

TYLER is trained in two distinct stages:

1. Stage 1 (Latent Synthesis Optimization): The backbone LLM is frozen. The Latent Synthesis Model and operator-specific components (query tokens, projection heads) are optimized. The model learns to synthesize latent tokens that effectively support the next visible token prediction. Heuristic rules based on the structure of CoT data (e.g., answer onset, step boundaries, code blocks) are used to assign operator types to specific positions in the training traces.
2. Stage 2 (Operator Invocation Optimization): The latent synthesis parameters are frozen. The backbone LLM is fine-tuned (via LoRA) to learn a budget-aware operator-invocation policy. This stage uses Group Relative Policy Optimization (GRPO). The reward function combines task performance with a penalty for exceeding a predefined latent-call budget ($B_O$). An operator-anchored auxiliary objective is introduced to stabilize learning at sparse invocation positions, ensuring the policy learns to select the correct operator type and timing.

3. Key Contributions

The paper outlines three primary contributions:

1. Formulation: It formulates latent reasoning as an online, typed, and budgeted computation problem, enabling LLMs to learn when to invoke latent computation, which operation to perform, and how much budget to allocate.
2. Framework (TYLER): It introduces a framework that interleaves visible decoding with three typed latent operators (Global Orientation, State Update, Procedural Abstraction), allowing for differentiated functional roles in latent space.
3. Empirical Validation: Extensive experiments demonstrate that TYLER improves accuracy over strong baselines, generalizes across diverse reasoning domains (math, science, theorems), and exhibits superior continual adaptation capabilities with lower forgetting compared to existing methods.

4. Experimental Results

Experiments were conducted on three backbone models (Qwen2.5-1.5B, SmolLM3-3B, Qwen3-4B) across four benchmarks: GSM8K, MATH-500, GPQA-Diamond, and TheoremQA.

• Performance Gains: TYLER achieves consistent improvements. On SmolLM3-3B, it improves average accuracy by 14.49 points over CoT and 4.30 points over the strongest baseline (MemGen). On Qwen3-4B, it improves by 9.22 points over CoT and 3.13 points over the strongest prior baseline (GRPO).
• Cross-Domain Generalization: Despite being trained primarily on mathematical traces (OpenR1-Math), TYLER generalizes effectively to scientific (GPQA) and theorem-oriented (TheoremQA) reasoning, suggesting the learned latent operators transfer across domains.
• Continual Learning: In a sequential adaptation setting (Code $\to$ Science $\to$ Math $\to$ Theorems), TYLER achieves the highest final performance with the lowest forgetting score ($F=2.3$), outperforming SFT, GRPO, and MemGen. This indicates a better stability-plasticity trade-off.
• Ablation Studies:
  • Removing the "typed" nature (sharing operators) or swapping operator roles significantly degrades performance, confirming the necessity of functional differentiation.
  • Random or fixed invocation schedules perform worse than the learned policy, proving the value of adaptive routing.
  • The "operator-anchored" loss is crucial for stabilizing performance in sparse invocation scenarios.
• Efficiency: TYLER achieves higher accuracy with lower latency and fewer total tokens compared to CoT and other latent baselines. Increasing the latent budget beyond 5 calls yields diminishing returns.

5. Significance and Claims

The paper claims that TYLER represents a shift from viewing latent reasoning merely as a compressed form of CoT to viewing it as a computation mechanism that allows models to choose between visible decoding and silent, specialized computation.

• Functional Specialization: The results suggest that latent operators can acquire distinct, differentiated functional roles (planning, state tracking, abstraction) rather than collapsing into a homogeneous representation.
• Adaptive Resource Allocation: The framework successfully demonstrates that an LLM can learn to allocate its internal computational budget dynamically based on task difficulty and reasoning stage, rather than relying on rigid schedules or external triggers.
• Robustness: The method shows robustness across different model scales and domains, maintaining performance gains even when the training data is domain-specific (math) but the evaluation is multi-domain.

The authors note limitations, including the reliance on heuristic boundary supervision in Stage 1, the current evaluation on reasoning benchmarks with verifier-style evaluation, and the reduced auditability of "silent" reasoning compared to explicit CoT. They position TYLER as a research contribution toward controllable latent computation rather than a deployed system.