Budget-Xfer: Budget-Constrained Source Language Selection for Cross-Lingual Transfer to African Languages

1. Problem Statement

Cross-lingual transfer learning is essential for enabling Natural Language Processing (NLP) in low-resource languages, particularly in Africa. A critical challenge is source language selection: determining which source languages to use for training and how to distribute a fixed annotation budget among them.

The Methodological Gap:
Existing research on source selection suffers from a confounding variable: total training data quantity. Different selection strategies often result in different total numbers of training examples because source languages have varying dataset sizes. This makes it impossible to isolate whether performance gains are due to the choice of languages or simply the amount of data. While some prior work (e.g., Lim et al.) equalized total data size, they often used uniform allocation or focused on mechanism rather than optimization.

The Core Question:
Given a fixed annotation budget $B$ (number of sentences), which source languages should be selected, and how should the budget be allocated across them to maximize downstream task performance?

2. Methodology: The Budget-Xfer Framework

The authors introduce Budget-Xfer, a framework that formulates multi-source cross-lingual transfer as a budget-constrained resource allocation problem.

2.1 Problem Formulation

Given a target language $t$, a pool of candidate sources $S$, and a fixed budget $B$, the goal is to find an allocation vector $\mathbf{a} = [a_1, ..., a_n]$ that maximizes performance $P(t|\mathbf{a})$ subject to:
$$\sum_{i=1}^{n} a_i = B, \quad a_i \geq 0$$
To approximate source value without exhaustive search, the framework uses cosine_gap (derived from FLORES-200 parallel data) as a utility proxy for embedding similarity between source and target languages.

2.2 Allocation Strategies

The study compares four distinct strategies (with $K=5$ for multi-source approaches):

1. All-from-Best (Single-Source Baseline): Allocates the entire budget to the single source language with the highest similarity to the target.
2. Top-K-Proportional: Selects the top $K$ most similar sources and allocates the budget proportionally to their similarity scores.
3. Random-K: Selects $K$ sources uniformly at random and splits the budget equally among them. This serves as an uninformed baseline.
4. Diversity-Aware: Iteratively selects sources to maximize a score that balances similarity to the target and diversity (penalizing redundancy) among selected sources. Budget is then allocated proportionally to raw similarity.

Note: A "cap-and-redistribute" procedure ensures that if a selected source has fewer available sentences than the allocated budget, the excess is redistributed to other sources, guaranteeing exact budget utilization ($B$) across all strategies.

2.3 Experimental Setup

• Tasks: Named Entity Recognition (NER) using MasakhaNER 2.0 and Sentiment Analysis using AfriSenti.
• Target Languages: Hausa, Yoruba, and Swahili (representing diverse language families).
• Models: Serengeti (517 African languages) and AfroXLMR.
• Budgets: Low and High levels (e.g., 10K/15K for NER; 5K/10K for Sentiment).
• Scale: 288 main experiments (2 tasks × 3 targets × 2 budgets × 2 models × 4 strategies × 3 seeds).

3. Key Contributions

1. Fair Comparison Framework: The first systematic comparison of allocation strategies under strictly controlled data conditions, isolating the effect of language selection from data quantity.
2. Diversity-Aware Strategy: Introduction of a selection method that explicitly balances source-target similarity with inter-source redundancy.
3. Actionable Empirical Findings: Large-scale experiments (288 runs) revealing that multi-source transfer is superior due to a structural bottleneck in single-source approaches, and that the value of similarity-based selection is highly task-dependent.

4. Key Results

4.1 Multi-Source vs. Single-Source Transfer

• Finding: Multi-source strategies significantly outperform single-source transfer (All-from-Best).
• Magnitude: Cohen's $d$ ranges from 0.80 to 1.98 across tasks and budgets.
• Root Cause: Structural Budget Underutilization. The single-source strategy is often constrained by the limited dataset size of the "best" source. On average, All-from-Best utilized only 57% of the allocated budget, whereas multi-source strategies utilized 100%.
  • Example: For Hausa sentiment analysis, the best source (Swahili) had only 1,810 sentences. At a 5K budget, the single-source strategy used only 36% of the budget, while multi-source strategies distributed the load to use the full 5K.
• Exception: When the best single source has sufficient data (e.g., Hausa for Swahili sentiment), single-source performance matches multi-source, confirming that the gap is driven by data scarcity, not an inherent flaw in concentrated training.

4.2 Comparison Among Multi-Source Strategies

• Finding: Differences between multi-source strategies (Top-K-Proportional, Random-K, Diversity-Aware) are modest and statistically non-significant.
• Observation: Once the decision to use multiple sources is made, the specific allocation rule (uniform vs. proportional) or selection criterion (similarity vs. random) has a minor impact compared to the decision to go multi-source.
• Win Rates: Random-K, Top-K-Proportional, and Diversity-Aware won roughly 28% of conditions each.

4.3 Task-Dependent Value of Similarity

The utility of embedding similarity as a selection proxy varies by task:

• NER (Token-level): Random selection outperformed similarity-based selection.
  • Reasoning: Similarity-based selection tends to cluster sources within the same typological family (e.g., Bantu languages). For NER, which requires recognizing diverse entity boundary patterns, exposure to linguistically diverse structures (provided by random selection) is more beneficial than concentrated exposure to similar structures.
• Sentiment Analysis (Sentence-level): Similarity-based selection outperformed random selection.
  • Reasoning: Sentiment relies on semantic properties that align well with embedding similarity. Linguistically similar sources provide more transferable signal for this task.

4.4 Budget Effects

• Increasing the budget from low to high yielded minimal performance gains (e.g., +0.012 F1 for sentiment).
• This suggests that for cross-lingual transfer with these models, 10K sentences (NER) or 5K sentences (Sentiment) often reach a saturation point, provided the data is distributed across multiple sources.

5. Significance and Implications

• Practitioner Guidance: Practitioners should always distribute annotation budgets across multiple source languages rather than focusing on a single "best" match. This avoids the structural bottleneck of underutilized budgets.
• Strategy Selection:
  • For Token-level tasks (NER): Prioritize linguistic diversity over similarity. Random selection or diversity-aware selection is preferable to similarity-based clustering.
  • For Semantic tasks (Sentiment): Prioritize linguistic similarity.
• Resource Efficiency: High-performance transfer can be achieved with relatively modest budgets if the allocation strategy avoids the single-source bottleneck.
• Robustness: These findings hold across different model architectures (Serengeti and AfroXLMR).

In conclusion, Budget-Xfer shifts the paradigm from "finding the best source" to "optimizing the distribution of a fixed budget," demonstrating that data quantity utilization is often the primary driver of performance gains in cross-lingual transfer to African languages.