Forecasting Supply Chain Disruptions with Foresight Learning

This paper introduces an end-to-end framework that trains large language models to generate calibrated probabilistic forecasts of supply chain disruptions, demonstrating that the resulting model significantly outperforms strong baselines, including GPT-5, in accuracy and reliability while fostering structured reasoning without explicit prompting.

The Gist

Imagine you are the captain of a massive cargo ship. Your job is to deliver goods across the world, but the ocean is unpredictable. Sometimes, a storm hits, a port closes, or a political dispute stops trade. These are supply chain disruptions.

The problem is that by the time you see the storm on the radar (official data), it's often too late to change your course. You need to predict the storm before it forms.

This paper introduces a new "super-weatherman" for global trade. Here is the story of how they built it, how it works, and why it's a game-changer.

1. The Problem: The "Silent Alarm"

Traditionally, companies wait for official reports to know if a supply chain is breaking. But these reports are like yesterday's newspaper—they tell you what happened, not what will happen.

Meanwhile, the news is buzzing with clues: a strike in a factory, a new trade law, or a drought affecting crops. But there is too much news, and it's messy. It's like trying to find a single needle in a haystack made of other needles. Standard computer programs get confused by this noise.

2. The Solution: Teaching an AI to "Think Like a Forecaster"

The researchers built a special version of a Large Language Model (LLM)—think of it as a super-smart robot that reads everything. But they didn't just ask it to read the news; they taught it a specific way of thinking called Foresight Learning.

The Analogy: The "Time-Traveling Student"
Imagine a student taking a test.

• Normal AI: You give the student the question and the answer key, then ask them to explain how they got it. They just memorize the answer.
• This New AI: You give the student only the news from October. You ask, "What will happen in November?" Then, you wait until November arrives, check the real answer, and say, "You were right!" or "You were wrong."

The AI learns by making predictions, seeing the real outcome, and adjusting its brain. It learns to connect the dots between a news headline today and a shipping delay next month.

3. How It Works: The "Crystal Ball" Training

The researchers fed the AI thousands of examples where they paired news articles with real disruption data.

• The Input: "Here is the news about a port strike in China today. Here is the current shipping index. What is the chance of a major disruption next month?"
• The Output: The AI doesn't just say "Yes" or "No." It gives a percentage (e.g., "There is a 35% chance").
• The Magic: The AI was trained to be calibrated. This means if it says "35%," it means exactly that. If it says "35%" on 100 different days, disruptions should happen on about 35 of them. Most other AIs are bad at this; they might say "90%" when the real chance is 10%.

4. The Results: Beating the Experts

The team tested their new model against:

1. Old methods: Just guessing based on history.
2. Top-tier AI: The most powerful general-purpose AI available (like GPT-5).

The Winner: Their custom-trained model crushed everyone.

• Accuracy: It predicted disruptions much more often than the others.
• Trust: Its probability estimates were honest. If it warned of a high risk, a disruption usually happened.
• Precision: When it gave a "high alert," it was right more often than any other system.

5. The Secret Sauce: How the AI "Thinks"

The most fascinating part isn't just that it got the right answer, but how it got there.

• Before Training: The AI acted like a news summarizer. It would say, "There was a strike, and there was a drought. So, maybe a disruption?" It was vague and didn't really do math.
• After Training: The AI started acting like a quantitative analyst. It began to:
  • Do mental math: "The usual volatility is X, so a jump of Y is rare."
  • Check the baseline: "Usually, this happens 10% of the time. Today's news bumps it up to 30%."
  • Update its thinking: "I thought it was 20%, but this new article about a labor deal changes it to 35%."

It learned to build a structured argument, weighing evidence like a human expert, without anyone explicitly telling it to "do math" or "check the baseline." It learned these habits just by trying to be a better forecaster.

6. Why This Matters

This is a big deal because it turns unstructured chaos (news articles) into actionable signals (probability percentages).

• For Companies: Instead of panicking when a disruption hits, they can get an early warning. "Hey, there's a 40% chance of a delay next month; let's stock up on inventory now."
• For Policymakers: They can see trouble coming and fix supply chains before they break.

The Bottom Line

The researchers built a "Time-Traveling Weatherman" for global trade. By teaching an AI to learn from its past predictions using real-world outcomes, they created a tool that doesn't just read the news—it understands the future risks hidden inside it. It's a leap from "guessing" to "calculated foresight."

Technical Summary

1. Problem Statement

The core challenge addressed is the anticipation of supply chain disruptions before they materialize.

• The Gap: Conventional indicators (e.g., trade data) are often delayed, revised, or only partially informative, creating a lag between the onset of a disruption and the availability of reliable signals for decision-makers.
• The Opportunity: News media provides a timely, unstructured stream of information regarding geopolitical tensions, labor disputes, and trade restrictions that precede measurable disruptions.
• The Difficulty: Extracting predictive signals from noisy, unstructured text and converting them into calibrated probabilistic forecasts is difficult. General-purpose Large Language Models (LLMs) struggle in this setting without task-specific adaptation, often failing to produce reliable probability estimates or reasoning that aligns with observed outcomes.

2. Methodology

The authors propose an end-to-end framework that trains LLMs to produce calibrated probabilistic forecasts using realized disruption outcomes as supervision.

A. Data Construction

• Sources: The dataset combines timestamped news articles (covering logistics, trade policy, geopolitics) with quantitative Supply Chain Disruption Indexes derived from granular international trade data (Liu et al., 2023).
• Task Formulation: A binary classification problem where the model predicts the likelihood of a "shock" in the following month ($t+1$).
  • Event Definition: A shock is defined as a month-over-month increase in the disruption index exceeding one standard deviation ($\sigma_e$) of historical changes for a specific entity (country or product).
  • Inputs: The model receives the current index value, the entity ID, and a context window of news articles available up to time $t$.
  • Labels: Binary outcomes ($y=1$ if shock occurs, $0$ otherwise) derived strictly from future realizations to prevent look-ahead bias.
• Dataset: ~5,400 forecasting questions (2022–2026), split chronologically into training (2022–2025) and testing (late 2025–early 2026) sets to ensure temporal validity.

B. Model Architecture & Training

• Base Model: GPT-OSS-120B, a 120B-parameter decoder-only transformer.
• Adaptation: The model is fine-tuned using Low-Rank Adaptation (LoRA) (rank 32) to specialize in the forecasting task without full-parameter updates.
• Training Framework (Foresight Learning):
  • The framework treats forecasting as supervised learning over temporally aligned text and future outcomes.
  • Objective: The model is optimized using a GRPO-style reinforcement learning objective. The reward function is the log-likelihood of the realized outcome given the predicted probability:
    $$r = y \log(p) + (1-y) \log(1-p)$$
  • This objective incentivizes both accuracy (predicting the correct class) and calibration (ensuring predicted probabilities match observed frequencies).

3. Key Contributions

1. New Forecasting Task: Introduces a novel task linking real-time, unstructured news to future supply chain disruption events, moving beyond simple sentiment analysis or structured feature extraction.
2. End-to-End Probabilistic Training: Demonstrates a method to train LLMs directly on raw text inputs to output calibrated probabilities, rather than using LLMs merely as feature extractors for downstream statistical models.
3. Superior Performance: Shows that the fine-tuned model significantly outperforms strong baselines, including frontier general-purpose models (e.g., GPT-5) and historical averages, in terms of accuracy, calibration, and precision.
4. Emergent Reasoning: Reveals that training under a forecasting objective induces structured probabilistic reasoning behaviors (e.g., base-rate anchoring, iterative uncertainty refinement) without explicit prompting.

4. Results

The study evaluates performance on a held-out test set using Brier Score, Brier Skill Score (BSS), Expected Calibration Error (ECE), and Precision@10%.

• Predictive Accuracy: The fine-tuned model achieved a Brier score of 0.0791, significantly outperforming GPT-5 (0.1203) and the untrained base model (0.1433).
• Calibration: The most significant improvement was in calibration. The fine-tuned model reduced the Expected Calibration Error (ECE) by nearly 80% (from 0.1740 to 0.0525) compared to the untrained model. This indicates the model's predicted probabilities closely match the actual frequency of disruptions.
• Precision: The model achieved a Precision@10% of 0.3478, compared to 0.0870 for GPT-5. This means the model's highest-confidence alerts are far more likely to be true disruptions, offering high practical utility for decision-makers.
• Reasoning Behavior Analysis: Automated evaluation of model outputs revealed that the fine-tuned model shifted from descriptive summarization to structured reasoning:
  • Base-rate anchoring: Increased from 9% to 50%.
  • Statistical modeling: Increased from 48% to 100%.
  • Uncertainty refinement: Increased from 33% to 100%.
  • The model began explicitly using thresholds, distributions, and iterative updates to refine its estimates.

5. Significance and Implications

• Decision-Ready Signals: The work demonstrates that LLMs can be trained to produce calibrated, decision-ready signals from unstructured data, bridging the gap between qualitative news analysis and quantitative risk management.
• Beyond Descriptive AI: It challenges the notion that LLMs are only useful for reasoning or extraction; they can be optimized for quantitative prediction under uncertainty when paired with the right training framework (Foresight Learning).
• Generalizable Framework: The approach suggests a general pathway for training domain-specific forecasting models in other fields where unstructured text precedes measurable outcomes (e.g., financial markets, public health, climate events).
• Transparency: The authors open-sourced the evaluation dataset and code, fostering reproducibility and further research in supply chain risk modeling.

In conclusion, this paper establishes that with appropriate temporal constraints and outcome-based supervision, LLMs can evolve from descriptive tools into robust, calibrated forecasting engines capable of anticipating complex supply chain disruptions.