Technical Summary: Do Reasoning Models Enhance Embedding Models?

Problem Statement

State-of-the-art text embedding models are increasingly constructed by adapting decoder-only Large Language Models (LLMs) via contrastive learning. Concurrently, a new class of "reasoning models" has emerged, trained via Reinforcement Learning with Verifiable Rewards (RLVR) to achieve superior performance in complex problem-solving. A natural hypothesis arises: does the enhanced reasoning capability of RLVR-tuned models translate into a superior semantic representation space when these models serve as backbones for embedding tasks? Intuitively, a model that "thinks" more deeply should structure semantic relationships more effectively. However, initial empirical observations suggest a paradox: embedding models initialized from RLVR-tuned backbones perform statistically identically to those initialized from base models after contrastive training, despite the reasoning models' demonstrated qualitative leaps in reasoning tasks.

Methodology

To resolve this paradox and diagnose the internal dynamics of these representations, the authors introduce Hierarchical Representation Similarity Analysis (HRSA), a framework extending Representational Similarity Analysis (RSA). HRSA decomposes model similarity into three nested levels of abstraction, each with specific invariance properties:

1. Representation Level: Focuses on the coordinate basis and feature correspondence. Metrics include Dimension-Wise Correlation (testing axis-aligned feature matching) and Orthogonal Procrustes Analysis (testing global linear equivalence via orthogonal transformations). This level is sensitive to rotations and permutations of the coordinate system.
2. Geometry Level: Focuses on the shape of the latent manifold, invariant to coordinate basis changes but sensitive to deformations. Metrics include Linear Centered Kernel Alignment (Linear CKA) for global geometry and k-Nearest Neighbors (k-NN) Overlap for local geometry preservation.
3. Function Level: Focuses on input-output mappings and downstream task solvability. This is evaluated using Cross-Model Linear Probes, where a linear readout trained on one model is tested on another to determine if the same decision boundaries are valid across representations.

Experimental Setup:
The authors conducted a systematic benchmarking study comparing base LLMs ($M_{base}$) against their RLVR-tuned reasoning counterparts ($M_{reason}$). They fine-tuned diverse pairs of these backbones into embedding models ($M_{Emb}^{base}$ and $M_{Emb}^{reason}$) using identical architectures and contrastive learning recipes (InfoNCE loss). Evaluation was performed on MTEB (Multilingual and Code) and BRIGHT benchmarks. The study also included Supervised Fine-Tuning (SFT) tuned models as a control to highlight differences between optimization strategies.

Key Results

1. Null Effect on Embedding Performance

Contrary to the hypothesis that enhanced reasoning improves embeddings, the results show a null effect. Embedding models initialized from RLVR-tuned backbones achieve performance parity with those initialized from base models across all benchmarks (MTEB Multilingual, MTEB Code, and BRIGHT). The performance gaps ($\Delta$) are near-zero, whereas SFT-tuned backbones often show significant deviations.

2. HRSA Findings: Manifold Realignment

HRSA reveals distinct structural behaviors between RLVR and SFT:

• Representation Level: RLVR largely preserves the coordinate basis. Dimension-wise correlations remain high, and Orthogonal Procrustes analysis yields matrices close to permutation matrices (low feature mixing). In contrast, SFT causes destructive mixing of features.
• Geometry Level:
  • Global Geometry: RLVR acts as a near-isometric transformation, preserving the global shape of the latent manifold (high Linear CKA).
  • Local Geometry: RLVR induces an irreversible reorganization of the local neighborhood structure (moderate k-NN overlap), suggesting a re-clustering of reasoning steps without altering the global semantic map.
• Function Level: Cross-model linear probes show strong transferability, indicating that task-relevant linear readout directions remain stable under RLVR.

3. Manifold Realignment Phenomenon

A critical finding is Manifold Realignment. When RLVR-tuned backbones are adapted into embedding models via contrastive learning, the training process drives a strong alignment between the base-initialized and reasoning-initialized models. This realignment effectively corrects the reversible coordinate basis drift observed in prolonged RLVR training, resulting in indistinguishable final embedding spaces.

Key Contributions

1. Systematic Benchmarking: The first controlled comparison demonstrating that current RLVR methods do not inherently improve embedding quality when used as backbones, despite their success in reasoning tasks.
2. The HRSA Framework: A unified, hierarchical diagnostic tool (Representation, Geometry, Function) that disentangles coordinate basis drift, manifold geometry changes, and functional readout stability. This framework offers a toolkit for future interpretability studies.
3. Discovery of Manifold Realignment: The identification that RLVR optimizes trajectories within an existing semantic landscape rather than fundamentally restructuring it. Specifically, RLVR preserves global geometry and linear readouts while reorganizing local neighborhoods, and subsequent contrastive learning can overwrite reversible drifts to realign the manifolds.

Significance and Claims

The paper claims that RLVR primarily functions as a trajectory optimization within an existing semantic landscape, rather than a fundamental restructuring of the landscape itself. Unlike Supervised Fine-Tuning (SFT), which can cause destructive mixing and global restructuring, RLVR refines the model's policy for traversing the latent manifold while preserving the rich world knowledge and global semantic relationships established during pre-training.

The authors posit that the "enhanced reasoning" of RLVR models is achieved by reorganizing local geometry (how nearby states relate) and optimizing decision trajectories, while leaving the large-scale coordinate system and linear readout directions largely intact. Consequently, for tasks relying on global semantic similarity (like embedding and retrieval), the RLVR advantage does not manifest as a superior embedding space but rather as a robust preservation of the base model's representational backbone. The paper suggests that this "Manifold Realignment" may be a general phenomenon in representation learning, extending beyond text to vision and audio, and proposes that similar behaviors could potentially be achieved via SFT augmented with geometry-aware regularization.