FireBench: Evaluating Instruction Following in Enterprise and API-Driven LLM Applications

This paper introduces FireBench, a new open-source benchmark comprising over 2,400 real-world enterprise and API-driven samples across six capability dimensions, designed to evaluate and improve instruction following in LLMs beyond traditional chat-based constraints.

The Gist

Imagine you hire a brilliant, hyper-intelligent assistant to help run your business. This assistant can write code, answer complex questions, and draft emails. However, in the real world of business, being "smart" isn't enough. You need the assistant to be obedient and precise.

If you ask the assistant to send a report in a specific spreadsheet format, it can't just write a nice poem about the data. If you tell it to ask for a customer's name before asking for their email, it can't mix up the order. If you tell it "don't mention the price," it can't accidentally slip it in.

This paper introduces FIREBENCH, a new "driving test" for AI models, specifically designed for these serious, real-world business situations.

Here is a breakdown of the paper using simple analogies:

1. The Problem: The "Chatbot" vs. The "Employee"

Most existing tests for AI are like talent shows. They ask the AI to write a funny story, use a specific number of words, or sound cheerful. These are great for a chatbot you talk to for fun.

But in a business (like a bank, a hospital, or a coding team), the AI is an employee.

• The Talent Show: "Write a poem about a cat in 3 paragraphs."
• The Employee Job: "Extract these 5 numbers from this 50-page legal document and put them into a JSON file. Do not add any extra text. If you don't know the answer, say 'I don't know'."

The paper argues that current tests are too focused on the "Talent Show" and ignore the "Employee Job." They don't check if the AI can follow strict, boring, but critical rules.

2. The Solution: FIREBENCH (The Business Driving Test)

The authors created FIREBENCH, a benchmark with over 2,400 test cases. Think of it as a simulator where the AI has to drive a delivery truck through a city with very strict traffic laws.

They test the AI on 6 specific skills that matter to businesses:

• 📝 The Format Police (Output Format Compliance):
  • The Test: "Give me the answer in a box, but the box must look like this specific shape."
  • The Analogy: Imagine a robot arm that needs to pick up a part. If the part is even 1 millimeter off-center, the machine breaks. The AI must output data in a format that a computer program can read perfectly. If the AI adds a comma or a space where it shouldn't, the whole system crashes.
• 🗣️ The Script Reader (Ordered Responses):
  • The Test: "Ask the customer for their name, then wait. Then ask for their address, then wait."
  • The Analogy: Like a waiter who must take an order in a specific sequence. If the waiter asks for the dessert before the appetizer, the kitchen gets confused. The AI must follow the script step-by-step without skipping ahead.
• 📊 The Sorter (Item Ranking):
  • The Test: "Here is a list of 100 products. Show me the top 5 most expensive ones, exactly as they appear in the list."
  • The Analogy: Like a librarian who must pull the top 5 books off a shelf based on a specific rule. The AI can't just guess; it has to sort the data perfectly and copy it exactly.
• 🛑 The "I Don't Know" Button (Overconfidence):
  • The Test: "Here is a question about a topic that isn't in your training data. Answer it."
  • The Analogy: A doctor who knows when not to prescribe medicine. If the AI doesn't know the answer, it must say, "I don't know," instead of making up a fake fact. In business, a fake fact can be dangerous.
• ✅ The "Must-Have" List (Positive Content):
  • The Test: "Write a contract that must include the phrase 'Force Majeure' and the date '2025'."
  • The Analogy: Like a packing list. If you are packing for a trip and forget your passport, the trip is ruined. The AI must include specific, mandatory ingredients in its answer.
• ❌ The "No-Go" Zone (Negative Content):
  • The Test: "Write a story, but do not use the letter 'e' and do not mention violence."
  • The Analogy: Like a strict diet. If you are on a "no-sugar" diet and you eat a candy bar, you failed. The AI must avoid specific words or topics entirely, even if it wants to include them.

3. The Results: The AI is Still Learning

The authors tested 11 of the smartest AI models available (like GPT-4, DeepSeek, Claude, etc.) using this new test.

The Shocking News:
Even the "smartest" AI models failed more than they passed.

• The best model only got about 74% of the questions right.
• Many models scored below 60%.

Key Findings:

1. One size does not fit all: A model might be amazing at formatting data (90% score) but terrible at sorting lists (30% score). You can't just pick the "best" model; you have to pick the right tool for the specific job.
2. Reasoning helps: Models that "think" before they speak (Reasoning models) were much better at sorting and ranking tasks than models that just guess immediately.
3. Formatting is still hard: Surprisingly, even simple formatting rules (like putting text in a specific box) trip up the AI. It seems the AI memorizes common formats but gets confused if you ask for a slightly weird variation.

4. Why This Matters

This paper is a wake-up call for companies. Just because an AI can write a poem doesn't mean it's ready to run your bank's database or your customer support line.

FIREBENCH is like a quality control inspector for businesses. It helps companies ask: "Is this AI actually safe to use for my specific needs?" before they let it loose in the real world.

The authors have made this test free and open-source, inviting everyone to help make it even better, ensuring that the AI of the future is not just smart, but also reliable and obedient.

Technical Summary

Here is a detailed technical summary of the paper "FIREBENCH: Evaluating Instruction Following in Enterprise and API-Driven LLM Applications."

1. Problem Statement

While Large Language Models (LLMs) have advanced significantly in natural language generation, their reliability in enterprise and API-driven settings remains a critical bottleneck. In these environments, strict adherence to output formats, procedural constraints, and content requirements is non-negotiable. A failure to follow instructions precisely (e.g., breaking a JSON schema, skipping a step in a workflow, or hallucinating facts) can cause downstream pipeline failures, security vulnerabilities, or compliance violations.

Existing instruction-following benchmarks (e.g., IFEval, FollowBench) primarily evaluate chat-assistant style constraints (e.g., word counts, tone, paragraph structure). These do not reflect the rigorous, deterministic, and structural requirements of enterprise applications, creating a gap between benchmark performance and real-world deployment reliability.

2. Methodology: FIREBENCH Design

The authors introduce FIREBENCH, a benchmark grounded in real-world enterprise usage patterns. It evaluates six core capability dimensions across 2,470 samples spanning information extraction, customer support, and coding agents.

Core Dimensions & Task Design:

1. Output Format Compliance (1,300 samples):

   • Rationale: Critical for programmatic parsing in agentic systems and RLVR.
   • Design: Tasks involve Question Answering (QA) and coding agents. Models must output answers in specific formats (JSON, XML, Markdown, \boxed{}) or adversarial variants (e.g., \boxed[ ]).
   • Verification: Fully programmatic.

2. Ordered Responses (200 samples):

   • Rationale: Essential for multi-turn customer support and data collection workflows.
   • Design: Simulates a multi-turn dialogue where an agent must collect 10–15 specific fields in a strict sequential order, asking exactly one question per turn.
   • Verification: Programmatic check against ground-truth ordering.

3. Item Ranking (200 samples):

   • Rationale: Ubiquitous in data-centric applications (e.g., sorting inventory, financial summaries).
   • Design: Models receive tables and must return the top-$N$ rows verbatim based on specific sorting criteria (equivalent to SQL ORDER BY).
   • Verification: Programmatic check of row order and content.

4. Overconfidence (370 samples):

   • Rationale: High-stakes applications require models to abstain when information is insufficient or confidence is low.
   • Design:
     • Challenging Tasks: Paired prompts (standard vs. uncertainty-aware) to test if models decline when unsure.
     • Insufficient Information: Long documents with unanswerable questions requiring explicit refusal.
   • Verification: Programmatic check for correct refusal vs. hallucination.

5. Positive Content Requirements (200 samples):

   • Rationale: Ensures mandatory inclusion of specific regulatory, stylistic, or structural elements.
   • Design: Prompts from Arena Hard 2.0 augmented with mandatory inclusions (e.g., "include a specific code pattern").
   • Verification: Evaluated via an LLM judge (GPT-4.1) using detailed rubrics.

6. Negative Content Requirements (200 samples):

   • Rationale: Enforces security policies and business logic by prohibiting specific content (e.g., "do not use temporary tables").
   • Design: Same base prompts as above, augmented with prohibitive constraints.
   • Verification: Evaluated via LLM judge; any violation is a failure.

3. Key Contributions

• Enterprise-Centric Benchmark: FIREBENCH shifts the focus from stylistic chat constraints to structural and procedural constraints vital for API and production pipelines.
• Comprehensive Evaluation Framework: It covers six distinct dimensions of instruction following, totaling over 2,400 samples with a mix of programmatic and LLM-based verification.
• Open-Source Resource: The benchmark is open-sourced (fire-bench.com) to facilitate model diagnosis and community contributions.
• Empirical Analysis of Frontier Models: The paper provides the first large-scale evaluation of 11 state-of-the-art models (including DeepSeek, GPT-4/5, Claude, Qwen, Kimi, and Llama) specifically on enterprise instruction following.

4. Results & Findings

The authors evaluated 11 closed and open-weight models. Key findings include:

• Significant Performance Gap: Even the best-performing model, DeepSeek-V3.1, achieved only 74.0% overall accuracy. No model surpassed the 75% threshold, indicating that reliable instruction following in enterprise settings is an unsolved challenge.
• High Variance Across Categories: Models exhibit inconsistent performance. For example, GPT-4.1 ranked 1st in Format (86.9%) and Content Requirements (>94%) but dropped to 5th/6th in Ranking (32.5%) and Overconfidence (38.6%). This suggests that a single "overall score" masks critical weaknesses in specific operational domains.
• Reasoning Models Outperform Non-Reasoning Variants:
  • Reasoning variants (e.g., Thinking models) consistently outperformed their Instruct counterparts.
  • The most dramatic improvement was in Item Ranking: GPT-5.1 Medium Thinking scored 93.0% vs. GPT-5.1 Instant's 16.0%. This indicates that explicit chain-of-thought processes are crucial for multi-step sorting and ordering tasks.
• Formatting Brittleness: Despite being a "mechanical" task, formatting remains a failure point. Models tend to memorize specific training formats (e.g., standard \boxed{}) rather than generalizing to arbitrary instructions. Performance dropped sharply (e.g., from 100% to ~53%) when presented with adversarial format variants.

5. Significance

• Deployment Readiness: FIREBENCH provides a practical tool for enterprises to assess whether a model is safe for production use, moving beyond "chat quality" to "pipeline reliability."
• Model Development: The benchmark highlights specific failure modes (e.g., poor ranking logic, formatting brittleness), offering a targeted framework for developers to diagnose and improve model architectures.
• Future Direction: The paper calls for the inclusion of composable constraints (satisfying multiple categories simultaneously) to better reflect the complexity of real-world enterprise workflows.

In conclusion, FIREBENCH demonstrates that while LLMs are powerful, they currently lack the robustness required for strict enterprise automation, necessitating careful model selection and further architectural improvements in instruction following.