🤖 machine learning

AI World Cup 2026: Benchmarking Large Language Models for End-to-End Football Tournament Prediction

This paper presents the "AI World Cup" benchmark, a standardized evaluation where ten large language models forecast the entire 2026 FIFA World Cup under identical conditions, revealing that GPT-5.5 Thinking outperformed others primarily due to superior knockout-stage predictions and that tournament-level forecasting success differs significantly from match-by-match accuracy.

Original authors: Jonaid Shianifar, Iias Faiud

Published 2026-08-05

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Jonaid Shianifar, Iias Faiud

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine a world where computers don't just answer questions about the past, but try to guess the future. This is the realm of forecasting, a branch of science where models act like crystal balls, trying to predict events before they happen. Unlike a trivia quiz where the answers are already written in a textbook, forecasting is like trying to predict the weather a month from now: you have to weigh conflicting clues, deal with missing information, and make a firm guess without knowing the outcome. The big question researchers are asking is: Can these super-smart computer brains, known as Large Language Models (LLMs), actually get better at this than humans? It matters because if we can teach machines to forecast complex events accurately, we could use them to predict everything from stock markets to natural disasters, not just sports. But to know if they are truly good at it, we need a fair test where everyone plays by the exact same rules.

Enter the AI World Cup 2026, a massive, real-time experiment that turned the biggest football tournament on Earth into a giant science project. The researchers set up a "battle royale" for ten different AI assistants. They gave every single AI the exact same snapshot of the tournament data, the exact same instructions, and asked them to do one thing: predict the entire tournament before a single ball was kicked. They had to guess the score of every group-stage match, figure out who would advance, and draw out the entire "bracket" (the path to the final) all the way to the champion. It was like asking ten different students to fill out a massive, 104-match tournament bracket sheet, but with the twist that the sheet had to be perfectly consistent—if you picked a team to win a game, you had to pick them to win the next one too.

The results were a wild ride that revealed a surprising truth about how these AIs think. When the dust settled and the real 2026 World Cup finished, the model named GPT-5.5 Thinking took the top spot with 744 points, followed closely by GPT-5.5 (717 points), Gemini (699), and Qwen 3.7 (687). But here is the twist: the winner wasn't the one who was best at guessing the score of individual games. In fact, the model that got the most individual match outcomes right, Claude Sonnet 4.6, only finished sixth overall.

Why did the rankings flip? The paper suggests that the secret sauce wasn't knowing who would win a single Tuesday night game, but rather keeping a coherent story for the whole tournament. The scoring system was heavily weighted toward the "knockout" stage (the elimination rounds). The data showed a very strong link (a correlation of 0.986) between a model's total score and how well it predicted the knockout path. In contrast, there was almost no link between total score and how well they predicted the early group-stage matches. It's like a detective who solves the final murder perfectly but gets the first few clues wrong; in this tournament, getting the final story right mattered more than getting the opening scenes perfect.

The paper also found some other fascinating quirks. For instance, the AI that was most confident in its answers (Mistral Medium 3.5) wasn't the most accurate at all. Confidence and accuracy were basically unrelated, with a correlation of -0.060, suggesting that just because an AI says it's sure doesn't mean it's right. Another surprise was the champion prediction: GPT-5.5 Thinking was the only model to pick Spain as the winner, and in a twist of fate, Spain actually won the real tournament, beating Argentina 1–0. This single correct guess, combined with a solid path through the knockout rounds, helped it leapfrog the others.

Ultimately, this paper suggests that predicting a whole tournament is a different skill than predicting a single game. It's not just about being a good football analyst; it's about being a good storyteller who can keep a consistent narrative from the first whistle to the final trophy. While the researchers are careful to say this is just one experiment with ten models and that the results depend heavily on how they decided to award points, the findings offer a clear roadmap for the future: if we want to test AI on big, complex predictions, we need to stop looking at them match-by-match and start judging them on how well they can see the whole picture.

Technical Summary: AI World Cup 2026

Problem Statement

Large Language Models (LLMs) are increasingly tasked with forecasting real-world events, yet comparative evaluation remains difficult due to inconsistent information inputs, varying tool usage, and disparate evaluation rules. Static benchmarks often suffer from data contamination, whereas prospective forecasting requires models to reason with incomplete evidence and commit to predictions before an event occurs.

The 2026 FIFA World Cup presents a uniquely demanding forecasting problem. With 48 teams and 104 fixtures, predicting the entire tournament requires more than isolated match-level judgment; it demands a consistent, long-horizon path from the group stage through the knockout bracket to the final. The challenge lies in evaluating whether LLMs can maintain a coherent tournament narrative where a single unexpected result can invalidate subsequent bracket predictions.

Methodology

The AI World Cup project established a controlled, reproducible benchmark to compare ten LLM-based assistants. The design adhered to eight methodological principles: identical prompts, a single data snapshot, manual submission via consumer interfaces, preservation of raw responses, structured validation, gradual evaluation, a transparent leaderboard, and separation of search-enabled systems where possible.

Experimental Design

Participants: Ten submissions from seven providers (OpenAI, Google, Qwen, DeepSeek, Anthropic, Mistral AI, Perplexity, xAI). Labels reflect specific assistant configurations rather than timeless model families.
Input: A single generated prompt provided a tournament snapshot and required a JSON output containing:
- Predictions for all 72 group-stage matches (score, outcome, winner, confidence, reasoning).
- Predicted rankings for all 12 groups.
- A complete knockout-stage bracket.
- Predictions for final placings (1st–4th) and individual awards.
Scoring System: An additive points system comprising three components:
1. Group-Stage Match Points: Awarded for exact scores (5 pts), correct outcomes (3 pts), correct winners (2 pts), and correct goal differences (1 pt).
2. Group-Standing Points: Awarded for correct group winners, top-two sets, qualifiers, and exact rank positions.
3. Knockout and Final-Placing Points: Awarded for correctly predicting a team's progression through each round (Round of 32 to Final) and for correct final placings (Champion: 20 pts, Runner-up: 10 pts, etc.).
Evaluation: All 104 matches were played. Results were audited against a frozen data snapshot (commit f83ea90) to ensure transparency. Correlations were calculated between score components and total scores.

Key Contributions

The paper provides:

A Completed Full-Tournament Evaluation: A head-to-head comparison of ten LLM assistants using a unified prompt, data snapshot, and scoring implementation.
Independent Audit: A decomposition of the final leaderboard into stage-specific accuracy, confidence analysis, and result provenance.
Evidence of Scoring Dynamics: Demonstration that knockout-bracket performance, rather than group-stage match prediction, was the primary driver of the composite ranking.
Comparative Analysis: A comparison with three contemporaneous 2026 World Cup benchmarks (WorldCupArena, WC2026-Agents, LLM-SoccerArena), highlighting differences in forecast timing and output granularity.
Reproducibility Roadmap: A set of recommendations for future benchmarks, including pre-registration, repeated independent runs, and probabilistic forecasting.

Results

Overall Performance

The final leaderboard was dominated by GPT-5.5 Thinking, which scored 744 points, followed by GPT-5.5 (717), Gemini (699), and Qwen 3.7 (687).

Champion Prediction: GPT-5.5 Thinking was the only model to correctly predict Spain as the champion (defeating Argentina 1–0).
Knockout Dominance: The final ranking was strongly correlated with knockout points ( $r = 0.986$ ). In contrast, total scores showed negligible correlation with group-stage match points ( $r = 0.055$ ) or group-standing points ( $r = -0.103$ ).
Variance: Knockout scores exhibited high variance (0 to 242 points), whereas group-stage match scores (246–270) and group-standing scores (221–264) were relatively clustered.

Divergent Leaders

Different models excelled in different metrics, indicating that "best" depends on the specific forecasting skill being measured:

Match Outcome Accuracy: Claude Sonnet 4.6 achieved the highest group-stage outcome accuracy (63.89%, 46/72 matches) but finished sixth overall due to poor knockout bracket consistency.
Exact Score Accuracy: Qwen 3.7 and Perplexity Pro predicted the most exact scorelines (10 each).
Knockout Path: GPT-5.5 Thinking led the knockout component by a significant margin (242 points vs. 196 for the next best), overcoming a deficit in group-stage performance to secure the top rank.

Confidence vs. Accuracy

Self-reported confidence showed no relationship with accuracy. The correlation between mean confidence and group-stage outcome accuracy was negligible ( $r = -0.060$ ), and between confidence and total score was similarly low ( $r = -0.067$ ). The highest-confidence model (Mistral Medium 3.5) was not the most accurate, and the winner (GPT-5.5 Thinking) reported one of the lowest average confidence values.

Significance and Claims

The paper claims that forecasting a complete tournament tests a different capability than predicting matches one at a time. The results suggest that:

Bracket Consistency is Critical: A coherent long-range forecast is distinct from local match accuracy. An early bracket error can invalidate subsequent predictions, making the knockout component the primary differentiator in composite rankings.
Scoring Design Drives Rankings: The leaderboard is heavily dependent on the weighting of the scoring system. In this design, the high-variance knockout component overshadowed the more stable group-stage components.
Limitations of Current Benchmarks: The study highlights that single-run, consumer-interface submissions introduce sampling variation and that self-reported confidence is not a reliable proxy for calibrated probability.
Future Directions: The authors argue for future benchmarks to utilize pre-registered protocols, repeated independent runs, probabilistic outputs (Brier scores), and strong non-LLM baselines (e.g., betting markets) to provide a more rigorous test of long-horizon forecasting.

The paper concludes that while GPT-5. Thinking produced the best realized tournament path under this specific scoring rule, the results do not prove a single model "understands football" better than others; rather, they demonstrate how different models excel at different facets of the forecasting problem (local accuracy vs. global consistency).