Technical Summary: Generative Recursive Reasoning Models (GRAM)

Problem Statement

Future neural reasoning systems face a central challenge: how to implement extended computation efficiently. While large autoregressive models scale reasoning by extending sequence generation (e.g., Chain-of-Thought), Recursive Reasoning Models (RRMs) offer an alternative by refining a persistent latent state through repeated application of shared transition functions. This approach decouples reasoning depth from parameter scale and output length.

However, existing RRMs (e.g., HRM, TRM) are fundamentally deterministic. Given the same input and initialization, they follow a single latent trajectory and converge to a single prediction. This determinism collapses the space of plausible reasoning paths into a single attractor, failing to maintain uncertainty, consider alternative hypotheses, or explore multiple solution strategies. This is particularly limiting for problems with intrinsic ambiguity, multiple valid solutions, or where a single refinement path may become trapped in suboptimal reasoning trajectories. The core question addressed is: Can recursive latent computation support probabilistic, generative, multi-hypothesis reasoning while preserving the efficiency of compact recurrent models?

Methodology: Generative Recursive Reasoning Models (GRAM)

GRAM introduces a framework that transforms recursive latent reasoning into probabilistic multi-trajectory computation. It treats the reasoning process as a stochastic latent trajectory, enabling the model to maintain multiple hypotheses and explore alternative solution strategies.

1. Architecture

GRAM models the conditional distribution $p_\theta(y | x)$ by marginalizing over stochastic latent reasoning trajectories. The architecture consists of an encoder, a recursive core, and a decoder, organized into two nested loops:

• Inner Loop (Low-Level Refinement): Within a single transition, a low-level latent component $l$ is refined $K$ times using a deterministic function $f_L$, conditioned on the high-level state $h$ and input embedding $e_x$.

• Outer Loop (High-Level Transition): The high-level component $h$ is updated once per transition. Unlike deterministic RRMs, GRAM defines this update as a stochastic transition:

  1. A deterministic update $u_t$ is computed: $u_t = f_H(h_{t-1}, l_t; \theta)$.
  2. A learnable stochastic guidance $\epsilon_t$ is sampled from a state-dependent Gaussian: $\epsilon_t \sim \mathcal{N}(\mu_\theta(u_t), \sigma^2_\theta(u_t)I)$.
  3. The new state is $h_t = u_t + \epsilon_t$.

  This design allows the model to capture uncertainty and steer the trajectory via the mean $\mu_\theta$ while controlling exploration via the variance $\sigma^2_\theta$. The low-level refinement remains deterministic, while stochasticity is injected only at the high-level abstract state.

• Recursive Structure: The computation proceeds through $N_{sup}$ supervision steps. The terminal state of one step serves as the initial state of the next, forming a full recursive chain.

2. Training

GRAM is trained using amortized variational inference to maximize the Evidence Lower Bound (ELBO).

• Latent Variable Formulation: The model marginalizes over latent trajectories $\tau$. The ELBO is optimized with respect to generative parameters $\theta$ and variational parameters $\phi$.
• Deep Supervision: To provide dense learning signals, the model applies deep supervision over $N_{sup}$ consecutive steps. Gradients are propagated only through the final transition of each supervision step (truncated backpropagation) to ensure memory efficiency, serving as a surrogate objective for the full ELBO.
• Unconditional Generation: By replacing the input with an empty conditioning embedding, the same recursive process defines an unconditional generative model $p_\theta(x)$.

3. Inference-Time Scaling

GRAM supports two axes of scaling:

• Depth: Using Adaptive Computation Time (ACT), trajectories can terminate at a learned halting depth.
• Width: The model samples multiple latent trajectories in parallel from the learned prior. Candidates are selected via majority voting or a Latent Process Reward Model (LPRM), which predicts the final quality of a trajectory from its latent state. This allows scaling inference without increasing sequential recursion length.

Key Contributions

1. Probabilistic Formulation: The authors formulate recursive reasoning as a latent-variable generative process where solutions are obtained by marginalizing over stochastic reasoning trajectories, rather than following a single deterministic path.
2. Width-Based Scaling: The paper introduces "width-based" inference-time scaling, enabling performance improvements by sampling multiple latent trajectories in parallel, complementing traditional depth-based scaling.
3. Empirical Validation: The authors provide evidence that this formulation yields architectural advantages over deterministic recurrent and recursive baselines, specifically in structured reasoning, multi-solution constraint satisfaction, and unconditional generation.

Experimental Results

GRAM was evaluated on controlled reasoning and generation tasks, comparing against deterministic baselines (Looped Transformers, HRM, TRM) and large reasoning models (LRMs) as external references.

• Structured Reasoning (Sudoku-Extreme, ARC-AGI): GRAM consistently outperforms deterministic recursive baselines. On Sudoku-Extreme, GRAM achieves 97.0% accuracy compared to 87.4% for TRM and 55.0% for HRM. On ARC-AGI-1, GRAM reaches 52.0%, surpassing TRM (44.6%) and HRM (40.3%). The authors attribute this to the ability to explore diverse reasoning trajectories.
• Multi-Solution Tasks (N-Queens, Graph Coloring): Deterministic models suffer from mode collapse as the number of valid solutions increases. GRAM maintains consistent accuracy regardless of solution count. On 8x8 N-Queens, GRAM achieves 99.7% accuracy and 90.3% coverage (unique valid solutions found), significantly outperforming deterministic models (max 36.1% coverage) and generative baselines like AR (84.8% coverage).
• Unconditional Generation:
  • Binarized MNIST: GRAM produces recognizable digits with Inception Scores (IS) and FID comparable to D3PM, whereas the deterministic TRM baseline exhibits severe mode collapse (FID 303.29).
  • Sudoku Generation: From an empty board, GRAM generates valid Sudoku boards with 99.05% validity using only 16 inference steps and 10.9M parameters, outperforming D3PM baselines that use up to 1000 steps and 55.1M parameters.
• Ablation Studies: Removing stochastic guidance leads to performance drops, particularly in multi-solution tasks. Removing stochasticity entirely causes complete failure (0% accuracy), confirming that the gains stem from the variational framework rather than mere randomness.

Significance and Claims

The paper claims that GRAM establishes probabilistic multi-trajectory recursion as a design principle for future recurrent and recursive reasoning architectures. By turning deterministic recursion into a generative process, GRAM enables:

• Robust Exploration: The ability to navigate complex problem spaces and escape local minima by maintaining multiple hypotheses.
• Efficient Scaling: A new axis for inference-time scaling (width) that bypasses the latency bottlenecks of depth-only scaling.
• Unified Modeling: The capability to function as both a conditional reasoning model ($p(y|x)$) and an unconditional generative model ($p(x)$) using the same recursive mechanism.

The authors note that while GRAM demonstrates potential in structured reasoning and generation, the sequential nature of deep supervision limits training efficiency compared to Transformers, posing a barrier to scaling toward larger foundation models. They also caution that plausible but invalid generations could be mistaken for verified solutions in high-stakes settings, necessitating rigorous validation and uncertainty calibration.