Optimization before Evaluation: Evaluation with Unoptimised Prompts Can be Misleading

1. Problem Statement

Current Large Language Model (LLM) evaluation frameworks (e.g., BigBench, LM Eval Harness, HELM) predominantly rely on static prompt templates. In these frameworks, a single, unoptimized prompt is applied uniformly across all models being evaluated.

The authors argue that this approach is fundamentally flawed for application-centric development because:

• Prompt Quality is a Variable: Prompt quality and style significantly impact a model's instruction-following capabilities and overall performance.
• Misleading Rankings: By treating the prompt as a fixed constant, benchmarks fail to capture the maximum potential performance of a model when paired with an optimized prompt.
• Industry Disconnect: While academic benchmarks use static prompts, industry practitioners routinely use Prompt Optimization (PO) techniques to tune prompts for specific models and tasks to maximize performance.

The core hypothesis is that evaluating models with unoptimized prompts leads to incorrect model rankings and suboptimal model selection for real-world tasks.

2. Methodology

The authors conducted experiments comparing standard static prompt evaluations against evaluations where prompts were optimized for each specific model.

Experimental Setup

• Models: Five leading LLMs (three closed-source multi-modal, two open-weight text-only) were evaluated. Model names were anonymized (Model A–E).
• Datasets: A mix of public academic benchmarks and internal industry datasets:
  • Open-source: GSM8K (Math), OpenbookQA, MMLU (Multitask).
  • Internal: Digital Assistant Routing, Copilot Help Docs, Copilot Consultancy, Text-to-SQL, EDDE (Document Extraction).
• Optimization Techniques: Two types of Prompt Optimization (PO) were implemented using GPT-4o as the "critic" or optimizer:
  1. Instruction-Only Optimization: Using the TextGrad framework to optimize the system and task instructions ($I$).
  2. Instruction-with-Exemplar Optimization: Using the MIPRO (via DSPy) framework to optimize both instructions and few-shot examples ($I + E$).
• Evaluation Metric: Kendall's Tau ($\tau$) was used to measure the correlation between model rankings derived from static prompts versus optimized prompts.

3. Key Contributions

1. Empirical Evidence of Rank Instability: The paper provides the first large-scale evidence that Prompt Optimization drastically alters model rankings. Models that appear "best" under static prompts often drop in rank when optimized prompts are used, and vice versa.
2. Quantification of Sensitivity: The study demonstrates that different models exhibit varying levels of sensitivity to prompt optimization depending on the task complexity.
3. Methodological Recommendation: The authors propose that Prompt Optimization should be a standard, mandatory step in application-centric model evaluation pipelines, rather than an optional post-processing step.

4. Key Results

A. Significant Rank Changes

• Weak Correlation: The correlation between rankings using static prompts and optimized prompts is very weak.
  • Instruction-only PO: Mean Kendall's $\tau \approx 0.23$.
  • Instruction-with-exemplar PO: Mean Kendall's $\tau \approx 0.36$.
  • A $\tau$ value near 0 indicates that the rankings are nearly uncorrelated, meaning the "best" model changes frequently.
• Specific Examples:
  • On GSM8K, static prompts suggested Model B was the best, but after instruction-only optimization, Model D became the top performer.
  • On MMLU, Model B was top with static prompts, but Model D took the lead after optimization.

B. Performance Improvements

• General Gain: PO generally improved the absolute performance scores of models across all datasets.
• Maximum Performance: For application-centric goals, PO allows practitioners to discover the true maximum performance ceiling of a model-prompt combination. For instance, Model A on the "Copilot Help Docs" dataset saw a 6.9% increase in maximum score after instruction-only optimization.

C. Task Sensitivity

• Complex vs. Simple Tasks: PO yielded the most significant benefits for complex, open-ended tasks (e.g., GSM8K, Copilot Consultancy).
• Diminishing Returns: For tasks where models already perform near-perfectly (e.g., OpenbookQA, Digital Assistant Routing), PO provided minimal gains or, in some cases, degraded performance due to over-optimization or redundant instructions.

D. Negative Cases (Failure Modes)

The paper also highlights instances where PO decreased performance:

• Format Confusion: In OpenbookQA, optimization added verbose reasoning instructions that confused the metric parser, causing a correct answer to be marked wrong.
• Exemplar Misinterpretation: In MMLU, an optimized prompt caused a model to treat few-shot examples as actual test questions, leading to a cascade of errors. This suggests a need for compatibility checks between the optimizer ("critic") and the target model.

5. Significance and Conclusion

The paper fundamentally challenges the status quo of LLM benchmarking. It argues that static prompts are insufficient for selecting the best model for a specific application.

• For Practitioners: Relying on public leaderboards with static prompts can lead to selecting a suboptimal model. To ensure the best ROI and performance, organizations must perform per-model prompt optimization before final evaluation.
• For Researchers: Future benchmarking frameworks must integrate automated PO as a core component of the evaluation pipeline to reflect real-world application scenarios accurately.

Conclusion: The authors conclude that "Optimization before Evaluation" is not just a best practice but a necessity. Ignoring prompt optimization leads to misleading benchmarks and potentially costly errors in model selection for production systems.