Rotate2Think: Geometric Priming via Orthogonal Rotation to Improve Language Model Reasoning

) using the model's inputs_embeds` interface.
4. Generate the final answer.

This process requires no gradient updates, no fine-tuning, and no additional inference compute beyond the single forward pass to extract the input embedding.

Key Contributions

1. Geometric Characterization of Reasoning: The paper establishes that input and thinking embeddings form nearly one-dimensional cones with non-aligned axes across diverse model families (Qwen, Phi, Gemma) and benchmarks. A single orthogonal rotation maps between them with high fidelity (cosine similarity > 0.96).
2. Rotate2Think Method: A training-free inference-time technique that fits an orthogonal Procrustes rotation on correctly solved examples and injects a synthetic thinking vector to prime the reasoning trace.
3. Cross-Domain Generalization: The method improves accuracy in 30 of 32 model-benchmark configurations across mathematics, science, and code tasks without target-domain data or per-benchmark refitting.
4. Zero-Shot Cross-Modal Transfer: A rotation fitted purely on text reasoning benchmarks successfully improves performance on the multimodal MATH-Vision benchmark, suggesting the input-to-thinking geometry is a property of the model rather than the input modality.

Experimental Results

Evaluated on four model families (Qwen3 4B, Phi-4 14B, Gemma-4 E4B, Gemma-4 31B) across five benchmarks (AIME 2025/2026, GPQA Diamond, BigCodeBench, MATH-Vision):

• Base Models: Injecting the synthetic vector into base models (Base+R2T) consistently improved accuracy, with gains up to +13 percentage points on AIME. In some cases (e.g., Gemma-4 E4B on GPQA), Base+R2T outperformed the same model in full reasoning mode.
• Reasoning Models: Augmenting reasoning models with the primer (Reasoning+R2T) yielded consistent improvements across all tasks, often without increasing token generation counts, suggesting more effective rather than longer reasoning.
• Multimodal Transfer: On MATH-Vision (testmini), the text-fitted rotation improved the base model's accuracy from 30.9% to 38.1%, surpassing the model's native full reasoning mode (34.9%).

Significance and Claims

The paper claims that Rotate2Think demonstrates that reasoning capacity may be latent and geometrically accessible. By intervening at a single point with a minimal perturbation (one rotated embedding), the method steers the model toward a representational regime where reasoning is more reliably expressed.

The authors argue that the success of cross-modal transfer supports the Platonic Representation Hypothesis, suggesting that representational geometry converges universally with model scale and that the "thinking space" is a structural property of the model independent of the input modality. The method offers a practical, lightweight building block for improving reasoning in both base and reasoning models without the cost of reinforcement learning or fine-tuning.

Limitations

The authors acknowledge several constraints:

• Dependency on Reasoning Variants: The method requires access to a reasoning-capable variant of the target model to extract the initial thinking embeddings for fitting.
• Task Specificity: Improvements were most pronounced in mathematical and scientific reasoning; gains on code generation were modest.
• Single-Token Injection: The current implementation injects only one synthetic token; the potential benefits of injecting a sequence of rotated embeddings remain unexplored.
• White-Box Requirement: The method requires access to the model's internal hidden states (specifically the last layer), precluding application to closed-source models where only token outputs are available.