Technical Summary: Task-Conditional Flow Matching for Balanced Multilingual Text Embedding Adaptation

Problem Statement

Multilingual text embedding models are frequently adapted to diverse downstream tasks (e.g., retrieval, classification, translation) using a single, unified training objective, typically contrastive learning. However, the authors argue that different task families require fundamentally different optimization strategies. Applying a uniform contrastive objective across heterogeneous tasks often leads to conflicting training signals, where improvements in one area come at the expense of others. Furthermore, standard contrastive learning struggles in multilingual settings due to the "false-negative" problem, where semantically similar sentences from different languages or contexts are incorrectly treated as negatives, fragmenting the continuous embedding space and disrupting the smooth geometry required for cross-lingual transformations.

Methodology: Task-Conditional Flow Matching (TCFM)

The paper proposes Task-Conditional Flow Matching (TCFM), a framework that selectively applies different optimization objectives based on the semantic characteristics of specific task families.

1. Task-Aware Objective Routing

TCFM categorizes training data into four task families and assigns specific objectives to each:

• Translation: Utilizes Flow Matching (FM). Since translation pairs provide naturally corresponding representations across languages, FM is used to model the smooth, continuous vector field transformations between source and target embeddings. This avoids the rigid separation of contrastive learning.
• Classification, Pair Classification, and Retrieval: These tasks optimize fundamentally different semantic relationships (e.g., shared labels, entailment, or asymmetric relevance) that do not naturally define transport trajectories. Consequently, TCFM routes these tasks to Multi-Positive Contrastive Loss and Hard-Negative Margin Repair, which are better suited for discriminative boundaries.

2. Core Components

• Flow Matching for Translation: The framework learns a deterministic velocity field to transport source embeddings to target embeddings. To stabilize optimization, the target velocity is interpolated with a direction predicted by a frozen teacher encoder. A "bounded transport" objective ensures the learned vector field influences the embedding space without causing large, unstable representation shifts.
• Teacher-Guided Representation Preservation: To prevent the adapted model from drifting away from the semantic structure of the pretrained encoder, TCFM employs a frozen teacher model. This includes:
  • Pointwise Loss: Ensures individual embeddings remain close to their original directions.
  • Relational Loss: Preserves the pairwise similarity structure of the batch.
• Hard-Negative Margin Repair: For tasks with explicit hard negatives (e.g., NLI, retrieval), a cosine-margin objective enforces strict separation between anchors and hard negatives, complementing the primary contrastive objective.
• Monolingual Consistency: A SimCSE-style self-consistency objective is applied to preserve within-language semantic integrity, preventing excessive cross-lingual alignment from degrading monolingual representation quality.

3. Three-Stage Training Curriculum

TCFM employs a progressive curriculum to balance cross-lingual alignment with task-specific adaptation:

• Stage 1 (Cross-Lingual Alignment): Focuses exclusively on translation data using Flow Matching and teacher preservation to establish a stable multilingual foundation.
• Stage 2 (Multi-Task Semantic Adaptation): Introduces classification and NLI datasets while maintaining a replay buffer of translation pairs (approx. 23% of the mix) to prevent catastrophic forgetting of cross-lingual alignment.
• Stage 3 (Retrieval Adaptation & Regularization): Incorporates asymmetric retrieval data and monolingual consistency objectives. The replay buffer is increased to ~41% of the data to further stabilize the model, while monolingual regularization resolves geometric interference.

Key Contributions

1. Task-Conditional Framework: The introduction of TCFM, which selectively applies Flow Matching to translation tasks while routing other tasks to more appropriate objectives (contrastive learning), rather than forcing a uniform objective across all tasks.
2. Empirical Validation of Task-Conditioning: The authors demonstrate that applying Flow Matching uniformly across all task families provides no additional benefit, validating the necessity of task-specific routing.
3. State-of-the-Art Performance: The framework achieves new state-of-the-art results on the Indic Massive Text Embedding Benchmark (Indic MTEB), consistently improving embedding quality across diverse tasks and generalizing across different model families (Gemma, Harrier, Qwen).
4. Curriculum Design: A three-stage curriculum that effectively balances the introduction of heterogeneous supervision with the preservation of pre-trained multilingual knowledge.

Results

Evaluated on the Indic MTEB benchmark across multiple model scales (ranging from 300M to 8B parameters), TCFM demonstrates significant improvements:

• Harrier-0.6B: Achieved a +3.59 point improvement in the overall Indic MTEB score, with a massive gain of +21.03 points in the Clustering task.
• Qwen3-Embedding-8B: Achieved a +1.97 point improvement overall, with significant gains in Clustering (+15.25) and Classification.
• Task Diversity: Improvements were observed across Bitext Mining, Classification, Clustering, Pair Classification, and STS. While most categories benefited, a small number of specific tasks showed modest decreases, highlighting the inherent difficulty of multi-objective optimization.
• Ablation Studies: Experiments confirmed that combining Flow Matching with teacher preservation yields complementary strengths: FM improves retrieval and clustering, while contrastive learning improves classification. Removing teacher preservation significantly degraded retrieval and STS performance.

Significance and Claims

The paper claims that task-aware optimization, which matches the loss function to the underlying task semantics, is a highly effective strategy for multilingual representation learning. The authors assert that:

• Applying a single contrastive objective across all training data is suboptimal for adapting multilingual embeddings.
• Restricting Flow Matching to translation tasks (where continuous transformations are natural) while using standard contrastive methods for retrieval and classification creates a more balanced and robust representation space.
• The proposed framework successfully generalizes across different model architectures and scales, suggesting that the benefits of task-conditional routing are not limited to specific model families.

The authors remain modest regarding limitations, noting that the evaluation is currently restricted to Indic languages, the geometric benefits of Flow Matching are empirically observed rather than formally proven, and the framework relies on high-quality parallel translation data, which may limit its applicability to zero-resource languages.