Improving LLM First-Token Predictions in Multiple-Choice Question Answering via Output Prefilling

1. Problem Statement

Large Language Models (LLMs) are frequently evaluated on Multiple-Choice Question Answering (MCQA) tasks using First-Token Probability (FTP). In this standard approach, the model is prompted with a question and options, and the answer is determined by selecting the option token (e.g., 'A', 'B', 'C', 'D') with the highest probability as the very first token generated.

While FTP is computationally efficient (avoiding full autoregressive decoding), the authors identify two critical failure modes that undermine its reliability:

• First-Token Misalignment: The model generates a full sentence or preamble (e.g., "I believe the answer is...") rather than a symbolic label. Consequently, the first token is not a valid option, making the evaluation impossible or misleading.
• First-Token Misinterpretation: The model generates a valid label as the first token, but it serves a grammatical or stylistic function rather than indicating the actual choice (e.g., "A possible answer is C"). Here, the first token 'A' is valid, but the intended answer is 'C'.

These issues lead to noisy accuracy metrics, poor calibration (mismatch between confidence and correctness), and distorted comparisons between models. Existing solutions, such as using external classifiers to parse free-form generation, are computationally expensive and slow.

2. Methodology: Output Prefilling

The authors propose Output Prefilling, a simple, zero-cost intervention that repurposes a technique originally used in AI safety (the "prefilling attack") to steer model behavior.

• Core Concept: Instead of relying solely on the prompt to instruct the model, the method injects a structured, benign natural-language prefix directly into the model's output stream (specifically, immediately after the assistant's start token).
• Implementation:
  • The prompt follows standard dialogue formats (e.g., <|user|>Question...<|assistant|>).
  • A fixed prefix is inserted before the model generates any new tokens. The default template used is: "Given the question and the possible options, my answer is: ".
  • This forces the model to treat the injected text as "already generated," conditioning its next token generation to continue logically from that prefix.
• Mechanism: By mechanically enforcing a specific output structure, the model is steered to generate the valid symbolic option (e.g., 'C') as the immediate next token, bypassing the tendency to generate preambles or grammatical fillers.
• Evaluation: The method retains the efficiency of standard FTP (scoring the first token) but applies it after the prefilling injection. It does not require fine-tuning, parameter modification, or external classifiers.

3. Key Contributions

1. Identification of Systemic Flaws: The paper rigorously quantifies the prevalence of first-token misalignment and misinterpretation across diverse LLMs, demonstrating that standard FTP is fragile even in instruction-tuned models.
2. Repurposing Prefilling: The authors introduce a novel application of the prefilling technique, transforming an adversarial safety bypass method into a robust evaluation tool for MCQA.
3. Zero-Cost Efficiency: The proposed method improves evaluation reliability without the computational overhead of full decoding or the latency of external LLM-based classifiers.
4. Comprehensive Benchmarking: The study evaluates the method across 14 diverse MCQA benchmarks (including MMLU, RACE, ARC, and SciQ) and 8 different open-source instruction-tuned LLMs (ranging from 7B to 14B parameters, including Llama-3, Qwen, Gemma, and Mistral families).

4. Experimental Results

The authors conducted extensive experiments comparing Standard FTP, Prompt Engineering (adding instructions to the input), and FTP + Prefilling.

• Accuracy Gains:
  • Prefilling consistently outperformed standard FTP and prompt engineering across all models and datasets.
  • Gemma-2-9B showed the most dramatic improvement, with an average accuracy gain of +25.9% over standard FTP.
  • Gemma-7B improved by +14.9%, and Phi-4-14B by +7.3%.
  • Even strong models like Llama-3.1-8B saw a +6.0% average gain.
• Comparison to Open-Ended Generation:
  • FTP with prefilling achieved accuracy levels comparable to, and often exceeding, expensive open-ended generation methods that rely on external classifiers (GPT-3.5, Llama-3-70B, xFinder) to parse answers.
  • This suggests prefilling effectively bridges the gap between efficient token-level scoring and the alignment of free-form reasoning.
• Full Vocabulary Analysis:
  • When allowed to generate from the full vocabulary (unconstrained), standard FTP often resulted in 0% validity for models like Gemma-7B (which always started with "The"). Prefilling restored validity rates to near 100% (e.g., Gemma-2-9B FTVR went from 0% to 99.8%).
• Calibration Improvements:
  • Prefilling significantly improved model calibration (measured by Adaptive Calibration Error, Brier Score, and Log Loss). Models became less overconfident in incorrect predictions, indicating that the method enhances the trustworthiness of probability estimates.
• Output Structure:
  • The method drastically reduced "Continuation Diversity" (the variety of tokens following the first answer token), indicating that models stopped generating verbose explanations and adhered strictly to the symbolic format.

5. Significance and Conclusion

This work demonstrates that Output Prefilling is a simple, robust, and highly effective strategy for evaluating LLMs in MCQA settings.

• Reliability: It solves the fundamental fragility of First-Token Probability, ensuring that the metric actually measures the model's reasoning capability rather than its formatting quirks.
• Efficiency: It offers a "free" performance boost in evaluation accuracy without requiring additional compute resources, fine-tuning, or complex decoding strategies.
• Generalizability: The method works across different model architectures, sizes, and alignment strategies, making it a universal best practice for MCQA benchmarks.

The authors conclude that relying on raw First-Token Probability without structural conditioning is insufficient for modern LLM evaluation. Output prefilling should be adopted as a standard procedure to ensure fair, accurate, and calibrated comparisons of model capabilities.