Technical Summary: AI Agents for Inventory Control: Complementarity of Human, LLM, and OR

Problem Formulation

The paper addresses the multi-period inventory control problem, a fundamental challenge in operations management, where a decision-maker must determine order quantities to balance uncertain demand over a finite time horizon. The specific scenario involves a single product with:

• State Dynamics: Existing inventory, in-transit orders, and historical demand.
• Uncertainties: Stochastic demand and stochastic lead times (including the possibility of lost orders where shipments never arrive).
• Objective: Maximization of total profit, defined as revenue from sales minus holding costs for remaining inventory.
• Constraints: Decisions are made sequentially with an observation delay of one period; lost sales are not backordered.

The core challenge is that traditional Operations Research (OR) algorithms, while theoretically grounded and computationally efficient, rely on rigid assumptions (e.g., stationary demand, known lead times) that often fail when demand distributions shift or when contextual information (e.g., seasonality, supply disruptions) is difficult to formalize. Conversely, Large Language Models (LLMs) offer flexible reasoning and world knowledge but lack the mathematical precision for optimal inventory calculations and may be poorly calibrated to cost structures.

Methodology

The study is divided into two parts: an algorithmic benchmark and a human-in-the-loop experiment.

Part 1: Algorithmic Benchmark (InventoryBench)

The authors construct InventoryBench, a benchmark comprising 1,320 instances:

• 720 Synthetic Instances: Based on 10 parametric demand pattern families (e.g., stationary, mean shifts, trends, seasonality, autocorrelation), combined with three lead-time regimes (zero, fixed, stochastic) and three critical fractiles (cost ratios).
• 600 Real Instances: Derived from historical sales data of 200 different H&M fashion products, incorporating real demand patterns and product descriptions.

Four decision mechanisms are evaluated using three leading LLMs (Gemini 3 Flash, Grok 4.1 Fast, GPT-5 Mini):

1. OR: A data-driven, base-stock policy using empirical demand estimates.
2. LLM: An agent making all decisions solely based on context and history, without OR input.
3. OR→LLM: The OR pipeline generates a recommendation; the LLM reviews it (with justification) and decides whether to override it.
4. LLM→OR: The LLM estimates uncertain parameters (demand mean, variance, lead time) based on context; these estimates feed into the base-stock policy for the final calculation.

Part 2: Human-in-the-Loop Experiment

A controlled classroom experiment with 69 participants playing a web-based inventory game in three collaboration modes:

• Mode A (OR→Human): Humans receive OR recommendations and make the final decision.
• Mode B (OR→LLM→Human): Humans receive OR recommendations, the LLM's justification/recommendation, and make the final decision.
• Mode C (OR→LLM + Human Oversight): The LLM makes autonomous decisions; humans provide high-level strategic guidance at periodic checkpoints.

Theoretical Framework for Complementarity

The paper introduces a theoretical framework to distinguish between complementarity at the population level and at the individual level.

• Population Level: The average team performance exceeds the maximum of the average human performance and the average AI performance.
• Individual Level: The collaborative performance of a specific individual exceeds the maximum of their solo performance and the AI's solo performance.
• Contribution: The authors derive a distribution-free lower bound (Theorem 1) for the proportion of individuals experiencing strictly positive complementarity. This bound is estimated using the Kolmogorov-Smirnov test between the distributions of solo and collaborative performance, enabling the estimation of individual benefits without observing the same person under both conditions.

Key Findings

Algorithmic Performance

• Complementarity: Hybrid methods consistently outperform isolated methods. OR→LLM achieves the best overall performance (normalized reward of 0.538), a 21% improvement over OR alone.
• Contextual Strengths: LLMs excel at detecting demand shifts, incorporating product-specific world knowledge (e.g., seasonality of swimwear), and identifying supply disruptions (lost orders).
• Mathematical Precision: OR algorithms provide the necessary precision for base-stock calculations, particularly under deterministic lead times.
• Calibration: Pure LLMs are less well-calibrated to cost structures (critical fractiles) than OR or hybrid methods. As LLM influence increases, responsiveness to the critical fractile decreases.
• Sensitivity to Lead Times: Under stochastic lead times (with lost orders), methods where the LLM makes the final decision (OR→LLM) are superior, as they can recognize and adapt to lost shipments, whereas OR-based methods fail to adapt to missing inventory in the pipeline.

Human-AI Interaction

• Superiority of Mode B: The OR→LLM→Human mode (Mode B) achieves the highest performance, significantly outperforming both Mode A (OR→Human) and Mode C (LLM + Oversight).
• Value of Human Judgment: Human collaboration modes outperform their automated counterparts (Mode A > OR; Mode B > OR→LLM), demonstrating that human judgment adds value beyond purely automated methods.
• Individual Complementarity: Applying the theoretical lower bound, the study finds that 30% to 60% of individuals experience strictly positive complementarity. This suggests that gains are not merely a selection effect (where weak humans follow AI and strong ones ignore it) but represent genuine synergy, where humans leverage AI reasoning to improve their own decisions.
• Mechanisms of Value: Humans add value by responding to demand trends, detecting anomalies (such as lost orders), and applying seasonal reasoning—capabilities that reflect LLM strengths over pure OR and suggest a "Co-Pilot" relationship where both agents reason contextually but with different error profiles.

Implications and Claims

The paper claims to provide systematic evidence that effective inventory management benefits from a complementary system rather than the replacement of one method by another.

• Synergy: OR provides structural discipline and mathematical precision; LLMs contribute world knowledge and contextual reasoning; humans provide situational judgment and oversight.
• Novelty in Human-AI Research: In contrast to earlier findings suggesting that human-AI collaboration often degrades performance or fails to surpass the best single component, this study demonstrates genuine complementarity in a sequential decision-making setting with rich contextual signals.
• Resource Contribution: The authors release InventoryBench (a benchmark with 1,320 instances) and the AI-Human Inventory Game (an open-source web interface) to facilitate future research in supply chain management and human-AI collaboration.

The authors conclude that while many directions remain open (e.g., richer agent architectures, broader participant pools), their results demonstrate that integrating OR, LLMs, and human judgment creates a decision pipeline that is robust to demand shifts and supply uncertainties, outperforming any single component in isolation.