OneSearch-V2: The Latent Reasoning Enhanced Self-distillation Generative Search Framework

1. Problem Statement

The paper addresses the limitations of OneSearch-V1, a generative retrieval (GR) framework deployed at Kuaishou for e-commerce search. While V1 offered end-to-end optimization and efficiency, it faced three critical bottlenecks:

1. Insufficient Complex Query Understanding: The model struggled with ambiguous, long-tail, or negation-based queries (e.g., "relieve fatigue, no supplements") because it relied on shallow semantic matching without deep reasoning capabilities.
2. Inefficient Personalized Intent Reasoning: Existing methods for Chain-of-Thought (CoT) reasoning required explicit token generation, causing prohibitive inference latency for online deployment. Furthermore, the model relied heavily on historical log-fitting, failing to uncover latent user intents (e.g., avoiding allergens for a specific user).
3. Fragile Reward System: The previous hybrid ranking framework relied on a separately trained reward model (RM) based on historical logs. This led to sampling bias, reward hacking, and an inability to adapt quickly to new queries or user intents, reinforcing long-tail distributional biases.

2. Methodology: OneSearch-V2 Framework

OneSearch-V2 introduces a Latent Reasoning Enhanced Self-distillation framework comprising three core innovations:

A. Thought-Augmented Query Understanding Module

• Mechanism: Instead of direct inference, the system uses a Large Language Model (LLM) to generate Keyword-based CoT (Chain-of-Thought).
• Process:
  1. Query Analysis: The LLM analyzes intent, category, attributes, and potential topics.
  2. Keyword Extraction: It extracts high-density keywords constrained by intent and attribute consistency.
  3. Preference Calibration: User profiles and historical behaviors are used to filter/augment keywords.
• Usage: These keywords serve as a semantic alignment corpus during training. Crucially, during inference, they are not generated by the model; instead, the reasoning capability is internalized into the model weights via self-distillation, avoiding latency overhead.

B. Reasoning-Internalized Self-Distillation Pipeline

• Goal: To transfer the explicit reasoning capability (derived from keyword-augmented inputs) into the model's parameters without requiring extra tokens or architectural changes at inference time.
• Architecture:
  • Teacher: Receives the query + keyword-based CoT.
  • Student: Receives only the raw query.
  • Shared Weights: Both share the same model parameters ($\theta$).
• Optimization:
  • KL Divergence: The student is trained to match the teacher's output distribution despite having less information.
  • Regularization: To prevent representation instability caused by information asymmetry, the authors apply R-Drop (for prediction consistency against dropout noise) and FGM (Fast Gradient Method for adversarial input robustness).
  • Result: The model learns to "intuit" the reasoning steps internally, enabling complex query understanding without generating explicit CoT tokens during serving.

C. Behavior Feedback Preference Alignment System

• Replacement: Replaces the separate Reward Model (RM) with a direct Behavior Feedback system using Reinforcement Learning (specifically GRPO).
• Composite Reward: Combines three signals:
  1. Relevance: Based on query-item semantic tiers.
  2. Posterior CTR: Calibrated click-through probability.
  3. Click & Order: Direct rewards for user interactions (purchases weighted higher than clicks).
• Token-Position Marginal Advantage (TPMA):
  • Standard GRPO assigns a uniform advantage to all tokens in a sequence. However, Semantic IDs (SIDs) have a strict hierarchical structure (Coarse Category $\to$ Fine Attributes).
  • TPMA-GRPO: Decomposes the reward into position-level marginal contributions. It introduces a Prefix Gate that suppresses gradients for downstream tokens if the prefix (coarse category) is incorrect. This ensures the model learns the hierarchy correctly before refining details.

3. Key Contributions

1. Keyword-based CoT for Query Understanding: A novel pipeline that extracts dense, intent-aligned keywords to guide training, overcoming the latency issues of full-text CoT.
2. Latent Reasoning via Self-Distillation: A method to internalize reasoning capabilities into model weights using information-asymmetric self-distillation, eliminating the need for additional inference tokens or architectural modifications.
3. Direct Behavior Feedback & TPMA: A reward system that bypasses separate reward models to use real user signals, coupled with a hierarchical credit assignment mechanism (TPMA) that respects the causal structure of SID generation.
4. Zero Latency Overhead: The entire framework achieves these gains without increasing inference latency or serving costs, as the reasoning is encoded in the weights.

4. Experimental Results

Extensive offline and online evaluations were conducted on the Kuaishou Mall search platform.

Offline Performance:

• Recall & Ranking: OneSearch-V2 achieved a 22.14% HR@10 (vs. 20.46% baseline) for orders and 10.48% MRR@10 (vs. 9.85% baseline).
• Ablation: Self-distillation was the single largest contributor to performance, followed by the combination of R-Drop, FGM, and Focal Loss.

Online A/B Testing (vs. OneSearch-V1):

• Item CTR: +3.98%
• Buyer Conversion Rate: +2.90%
• Order Volume: +2.11%
• GMV (Gross Merchandise Value): +3.45%
• Manual Evaluation: +1.65% improvement in Page Good Rate and +1.37% in Query-Item Relevance.

Granular Analysis:

• Long-Tail Queries: Showed the most significant improvement (+5.37% CTR), validating the model's ability to handle ambiguous queries.
• Cold Items: New items saw a 6.16% CTR lift, indicating better discovery capabilities.
• User Segments: Consistent gains across low, medium, and high-activity users.

5. Significance

• Industrial Scalability: OneSearch-V2 demonstrates that complex reasoning capabilities can be deployed in high-concurrency, low-latency e-commerce search environments without the computational cost of generating explicit CoT.
• Beyond Logs: By utilizing self-distillation and direct behavior feedback, the system mitigates the "information bubble" and overfitting to historical logs, allowing it to generalize to new intents and long-tail scenarios.
• Framework Advancement: It establishes a new paradigm for Generative Retrieval where reasoning is internalized rather than explicit, and preference alignment is driven directly by user behavior rather than proxy reward models.
• Open Source: The authors have open-sourced the code and data cases, contributing to the broader research community in generative search.