Improving Cross-Format Robustness in Language Models with Multi-Format Training

Technical Summary: Improving Cross-Format Robustness in Language Models with Multi-Format Training

Problem Definition
Large Language Models (LLMs) exhibit significant sensitivity to answer formats. A model may correctly answer a question presented as a Multiple-Choice Question (MCQ) but fail when the same underlying semantic content is presented as a True/False (TF), Fill-in-the-Blank (FIB), or Open-Ended (OPEN) query. This "cross-format inconsistency" suggests that current models rely on format-specific shortcuts rather than robust knowledge access. While prior work has largely focused on adjusting evaluation metrics to account for this sensitivity, this paper posits that format sensitivity is fundamentally a training problem that can be addressed through supervision.

Methodology
The authors propose a controlled framework to study and improve cross-format robustness through Multi-Format Supervised Fine-Tuning (SFT).

1. Data Construction:

   • Source: The study utilizes six widely used MCQ benchmarks (C-Eval, CMMLU, MMLU, MMLU-Pro, GPQA, SuperGPQA).
   • Reformulation: Items are screened for "reformulatability" (ensuring they can be converted without losing answer uniqueness or gradability). Valid items are then rewritten into four semantically aligned formats: MCQ, TF, FIB, and OPEN.
   • Paraphrasing: For each format, three paraphrases are generated.
   • Validation: A multi-stage pipeline involving four LLMs (DeepSeek-V3.1, Qwen3, Kimi-K2.5, GPT-OSS-120B) performs generation, independent review, and arbitration to ensure semantic alignment and scorability. Human validation confirms high quality (>90% on all criteria).
   • Corpus: The final dataset contains 11,842 source items, resulting in 142,104 instances across formats and paraphrases.

2. Evaluation Protocol:

   • The authors introduce an item-level metric system to disentangle solvability from consistency:
     • pass@4: The fraction of items solved in at least one of the four formats (measuring overall solvability).
     • pass4: The fraction of items solved in all four formats (measuring consistency).
     • Cross-Format Robustness: Defined as $pass4 / pass@4$. This ratio measures the probability that an item is solved in all formats given that it is solvable in at least one, effectively filtering out inherently unsolvable items.
   • Scoring: MCQ and TF use rule-based extraction with LLM-as-a-Judge fallback. FIB and OPEN rely on LLM judges (GPT-5, Gemini-2.5-Pro, Claude-Sonnet-4.6) with majority voting.

3. Training Regimes:
   The study compares several fine-tuning strategies on GLM4-9B and Llama-3.1 models:

   • Baseline: Untuned base model.
   • Full-Format SFT: Every training item is expanded into all four formats (4x data volume).
   • FormatMix (30%): Only 30% of source items are expanded into all four formats; the rest remain MCQ-only. This includes:
     • Random-FormatMix: Random selection of items.
     • Targeted-FormatMix: Selection based on the base model's cross-format inconsistency (items with high standard deviation in accuracy across formats).
   • MCQ-Matched: A control group with the same total training instances as the 30% FormatMix groups, but all data remains in MCQ format (isolating the effect of format diversity vs. training volume).

Key Results

• Multi-Format Supervision Improves Robustness: Full-Format SFT significantly increases cross-format robustness. On GLM4-9B, robustness improved by 35% relative to the baseline (from 0.1575 to 0.2126), and pass4 increased by 62%.
• Format Diversity is the Driver: The MCQ-Matched control (more training data, same format) improved MCQ accuracy but failed to improve pass4 or cross-format robustness. This indicates that the gains come from format diversity, not merely increased supervised updates.
• Efficiency of Partial Expansion: Expanding only 30% of the training set into multiple formats recovers the majority of the benefits seen in full expansion.
  • Targeted-FormatMix (selecting inconsistent items) yielded the highest pass4 and robustness scores among the 30% budget settings.
  • Random-FormatMix yielded slightly higher pass@4 (breadth), suggesting a trade-off between coverage and consistency.
• Generalization: These patterns hold across model families (GLM4 and Llama-3.1) and sizes (9B, 32B, 70B). While larger models possess higher native robustness, multi-format SFT still provides marginal gains, suggesting diminishing returns rather than redundancy.
• External Validation: On the BIG-Bench Hard dataset, multi-format training improved performance across MCQ, TF, and OPEN tasks, with Targeted-FormatMix performing particularly well on open-ended tasks compared to MCQ-only training.

Significance and Claims
The paper claims that format robustness is a trainable capability. The primary contribution is demonstrating that "lightweight multi-format augmentation" is a practical method to make LLMs less sensitive to answer formats without requiring changes to the base model architecture or exhaustive rewriting of the entire training corpus.

The authors argue that the key driver of robustness is format diversity in supervision, not just additional supervision volume. Furthermore, they suggest that targeting the most format-sensitive items (via inconsistency-based selection) is an effective heuristic for maximizing robustness gains under constrained compute budgets. This work reframes question-format sensitivity from a purely evaluation-side artifact to a solvable training objective, offering a pathway to more reliable LLM deployment in real-world scenarios where questions are rarely posed in a single, fixed format.