What Prompts Don't Say: Understanding and Managing Underspecification in LLM Prompts

1. Problem Definition: Prompt Underspecification

The paper identifies prompt underspecification as a critical, yet often overlooked, challenge in building reliable Large Language Model (LLM) applications.

• Definition: Underspecification occurs when essential requirements are omitted from a prompt, leaving multiple valid but inconsistent behavioral outcomes possible.
• The Core Issue: While developers often rely on LLMs to "fill in the gaps" using commonsense, this behavior is fragile.
  • Instability: Unspecified requirements are 2x more likely to regress (fail) when the underlying model or the prompt changes compared to explicitly specified requirements.
  • The "Over-specification" Trap: Simply adding all possible requirements to a prompt is not a viable solution. Due to limited instruction-following capabilities, LLMs struggle to process long, complex lists of constraints, leading to a 19% drop in accuracy when specifying 19 requirements versus fewer.
• Consequences: This leads to unpredictable user experiences (e.g., a travel advisor failing to warn about visa requirements or weather) and makes maintaining robust LLM applications difficult.

2. Methodology

The authors conducted a systematic empirical analysis involving three representative tasks: Trip Advisory, Product Description Generation, and Code Explanation.

• Data Curation:
  • Curated 60 plausible requirements per task (totaling 180+ requirements) using three elicitation methods: analysis of existing real-world prompts, brainstorming potential failure modes, and error analysis of model outputs.
  • Constructed 8,400 data points by generating prompts with different subsets of these requirements (using a cyclic design to ensure balanced coverage).
• Evaluation Framework:
  • Validators: Developed automated validators (Python scripts or LLM-as-a-judge) to verify if model outputs satisfied specific requirements. Human validation showed 95.6% agreement with LLM validators.
  • Models Tested: Evaluated across various models including Llama-3.3-70B, gpt-4o, and o3-mini.
  • Metrics: Measured satisfaction rates (accuracy) and variance (stability) across different prompt configurations and model updates.

3. Key Empirical Findings

The study yielded several counter-intuitive but critical insights:

1. LLMs Can Guess, But Unreliably:

   • LLMs successfully infer unspecified requirements 41.1% of the time.
   • However, this guessing is highly inconsistent. Different prompts or model versions yield vastly different results for the same unspecified requirement (standard deviation of 8.9% for unspecified vs. 0.8% for specified).
   • Regression Risk: When models are updated, unspecified requirements regress significantly more often. For example, 5.9% of unspecified requirements dropped in accuracy by >20% after a model update, compared to only 3.0% for specified ones.

2. The "Instruction Following" Bottleneck:

   • LLMs perform best when requirements are isolated.
   • As the number of specified requirements increases, average accuracy drops. Specifying 19 requirements caused accuracy to fall to 85.0% for gpt-4o and 79.7% for Llama-3.3-70B.
   • This suggests a trade-off: Over-specifying overwhelms the model, while under-specifying relies on unstable defaults.

3. Failure of Standard Optimizers:

   • Existing prompt optimizers (e.g., OpenAI's static optimizer, DSPy's COPRO) that do not account for specific requirements provide inconsistent or negligible improvements. They often fail to distinguish between requirements the model already follows by default and those that need explicit instruction.

4. Proposed Solutions

The authors propose a two-pronged approach to managing underspecification:

A. Requirements-Aware Prompt Optimization

Instead of blindly optimizing text, the authors introduce mechanisms that explicitly consider the requirement set:

1. COPRO-R (Requirement-Aware Feedback): Enhances existing optimizers by using requirement-specific validators as the feedback signal. This guides the optimizer to fix specific failures rather than optimizing for a generic "goodness" score.
2. Bayesian Optimizer (Selection Strategy): A novel approach that treats the selection of requirements as a hyperparameter optimization problem. It uses a Tree-structured Parzen Estimator (TPE) to determine which requirements to explicitly specify and which to leave implicit (relying on the model's default behavior).
   • Result: This strategy reduced token usage by 41–45% while improving accuracy by 4.8% on average. It effectively identifies redundant requirements (e.g., global formatting rules) that the model already follows.

B. Systematic Management Process

Beyond optimization, the paper advocates for an engineering lifecycle change:

• Proactive Discovery: Move beyond ad-hoc trial-and-error to systematic requirement elicitation (e.g., using synthetic data to find edge cases).
• Continuous Monitoring: Implement background validation of all tracked requirements (both specified and unspecified) to detect drift when models or prompts change.
• Validator Alignment: Ensure that the validators used to evaluate prompts are themselves robust and aligned with human intent.

5. Key Contributions

1. Characterization: First comprehensive empirical study quantifying the instability of unspecified requirements in LLMs.
2. Evidence of Fragility: Demonstrated that unspecified requirements are 2x more prone to regression across model updates than specified ones.
3. New Optimization Paradigm: Introduced Requirements-Aware Prompt Optimization, showing that optimizing what to specify is as important as how to specify it.
4. Engineering Framework: Proposed a structured process for the discovery, curation, and monitoring of requirements to support reliable LLM application development.

6. Significance

This paper shifts the paradigm of prompt engineering from a "craft" based on intuition to a systematic engineering discipline.

• For Developers: It warns against the "vibe check" approach (testing a few examples) and highlights the need for rigorous requirement management.
• For Researchers: It identifies a new optimization objective: balancing explicit specification with implicit inference to maximize stability and minimize token usage.
• For Industry: It provides a concrete mechanism (Bayesian requirement selection) to build more robust, cost-effective, and maintainable LLM applications that do not break silently when models are updated.

In summary, the paper argues that reliability in LLM applications cannot be achieved by either over-specifying (which confuses the model) or under-specifying (which relies on fragile defaults). Instead, it requires a "Goldilocks" approach driven by data-aware optimization and continuous monitoring.