Technical Summary: GeoMin - Data-Efficient Semi-Supervised RLVR via Geometric Distribution Modeling

1. Problem Statement

Reinforcement Learning with Verifiable Rewards (RLVR) has become a pivotal paradigm for enhancing the reasoning capabilities of Large Language Models (LLMs). However, the scalability of RLVR is severely constrained by the "annotation bottleneck": standard supervised approaches require costly ground-truth annotations, while unsupervised alternatives often suffer from "model collapse" due to reliance on noisy self-reward signals (e.g., majority voting, entropy) that reinforce false confidence without factual grounding.

Recent semi-supervised approaches, such as TraPO, attempt to bridge this gap by using a small labeled set to guide the selection of high-quality unlabeled data. However, these methods face a critical data-efficiency bottleneck. They rely on coarse performance heuristics (e.g., aligning individual trajectory pass-rates with collective averages), which fail to account for the natural fluctuations in individual instance optimization. Consequently, these methods exhibit a severe "recall bottleneck," utilizing only a tiny fraction (e.g., ~12.3%) of the available reliable unlabeled data, leaving the majority of valuable instances underutilized.

2. Methodology: GeoMin

The authors propose GeoMin, a two-stage framework that leverages the geometric properties of latent representations to efficiently mine high-quality unlabeled data. The core insight is that correct and incorrect reasoning rollouts exhibit a distinct geometric resonance in their directional distributions, which can be modeled to assess the reliability of self-reward signals.

Core Concept: Geometric Priors

The method models the hidden states of LLMs (specifically normalized last-token embeddings) as directional data on a hypersphere. It posits that correct and incorrect rollouts form distinct clusters that can be modeled using von Mises-Fisher (vMF) distributions.

• Geometric Resonance: As training progresses, the directional distributions of correct and incorrect rollouts naturally separate. Unlabeled correct rollouts align with the "correct" labeled distribution, while incorrect ones align with the "incorrect" distribution.

Stage 1: Supervised Anchoring

To establish a robust geometric prior, GeoMin first performs supervised training exclusively on a small labeled set ($D_l$).

• Distribution Modeling: It fits online vMF distributions ($\mu_c, \kappa_c$) for correct ($c=1$) and incorrect ($c=0$) classes across network layers.
• Boundary Disambiguation: The method identifies "ambiguous boundary samples" where the geometric alignment of rollouts contradicts their correctness (e.g., a correct rollout is geometrically closer to the incorrect distribution).
• Advantage Reweighting: To accelerate the emergence of geometric separation, the advantages of these boundary samples are amplified during the Group Relative Policy Optimization (GRPO) update. This sharpens the decision boundary in the representation space.

Stage 2: Geometry-Guided Sample Mining

In the second stage, the framework incorporates unlabeled data ($D_u$) for semi-supervised training.

• Dynamic Adaptation: The vMF distributions are continuously refreshed using the labeled subset to track the evolving representation space.
• Discriminative Layer Selection: The method quantifies layer-wise separability ($\Delta^{(l)}$) and selects the top-$K$ layers with the highest diagnostic fidelity for reliability assessment.
• Confidence Scoring: For each unlabeled instance, a confidence score ($s_i$) is computed by aggregating the directional affinities of its rollouts against the learned correct and incorrect vMF distributions.
• Adaptive Filtering: Instead of using a static threshold, GeoMin fits a Gaussian Mixture Model (GMM) to the distribution of confidence scores. It adaptively selects high-confidence samples (those belonging to the high-mean Gaussian component) to form a reliable pseudo-labeled set ($B^*_u$).
• Training: The final policy is optimized on the union of the original labeled set and the filtered reliable unlabeled set.

3. Key Contributions

1. Geometric Resonance Discovery: The paper identifies and formalizes the "geometric resonance" phenomenon, where the structural discrepancy between correct and incorrect rollouts can be decoded via global feature distributions, providing a robust prior for assessing pseudo-label reliability.
2. Novel Two-Stage Framework: GeoMin introduces a coordinated approach combining supervised anchoring (to establish discriminative boundaries) and geometry-guided mining (to adaptively select unlabeled data), overcoming the recall limitations of previous semi-supervised methods.
3. Data Efficiency: By modeling distributions rather than relying on macro performance heuristics, GeoMin achieves an F1 score of 89.0% in identifying reliable unlabeled samples, significantly outperforming the 21.4% F1 score of the TraPO baseline.

4. Experimental Results

Experiments were conducted on the Deepmath-103k dataset using Qwen3-8B-Base and other models (1.5B, 8B).

• Performance Gains: GeoMin outperforms the strongest semi-supervised baseline (TraPO) by +4.1% in In-Domain (ID) and +1.7% in Out-of-Domain (OOD) average accuracy.
• Annotation Efficiency: Remarkably, GeoMin trained with only 10% of the annotations surpasses the fully supervised baseline (trained on 100% data) by +1.2% in ID performance, while maintaining competitive OOD performance.
• Scalability: The method demonstrates consistent improvements across different model scales (1.5B to 8B) and architectures (Qwen vs. Llama).
• Efficiency: GeoMin achieves a 2.14× speedup in total training time compared to TraPO, as it avoids the wasteful multi-epoch warm-up phases required by trajectory-matching baselines.

5. Significance and Claims

The paper claims that GeoMin represents a significant step toward scalable, data-efficient RLVR. By shifting the focus from surface-level output probabilities to the geometric alignment of latent representations, the method unlocks the potential of vast unlabeled datasets without compromising training stability.

The authors assert that GeoMin effectively bypasses the annotation barrier, enabling models to achieve superior generalization with minimal human supervision. The work suggests that the "recall bottleneck" in semi-supervised RLVR is not inherent to the paradigm but stems from the reliance on coarse heuristics, which can be resolved through rigorous geometric distribution modeling.

Limitations Acknowledged:

• Validation was primarily conducted on models up to 8B parameters; performance on larger frontier models remains to be verified.
• The method's efficiency depends on the initial geometric discriminability of the model; models with poor initial representation quality may require extended Stage 1 training.
• Current evaluation is limited to textual and symbolic reasoning; generalization to multimodal or cross-lingual tasks requires further investigation.