💬 NLP

Large Language Models are Algorithmically Blind

This paper reveals that despite their extensive knowledge, large language models suffer from "algorithmic blindness," systematically failing to accurately predict or reason about computational processes and algorithmic performance, often performing no better than random guessing due to a fundamental gap between their declarative knowledge and procedural reasoning capabilities.

Original authors: Sohan Venkatesh, Ashish Mahendran Kurapath, Tejas Melkote

Published 2026-03-03

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Sohan Venkatesh, Ashish Mahendran Kurapath, Tejas Melkote

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Idea: The "Know-It-All" Who Can't Do Math

Imagine you have a friend who has read every book in the library. They can tell you the history of the Roman Empire, the plot of every movie ever made, and the definition of a "computer algorithm." They are incredibly knowledgeable.

Now, imagine you ask this friend: "If I use a specific tool to fix a specific broken car, how long will it take, and how likely is it to work?"

You expect them to give you a smart, educated guess based on the car's make and the tool's specs. Instead, they give you a wildly inaccurate answer. They might say, "It will take between 2 minutes and 200 years, and it will definitely work perfectly," even though the tool is known to fail on that specific car 90% of the time.

This paper argues that Large Language Models (LLMs) are exactly like that friend. They have read about algorithms, but they cannot actually predict how well an algorithm will perform on a new problem. The authors call this "Algorithmic Blindness."

The Experiment: The "Causal Discovery" Test

To prove this, the researchers set up a rigorous test using a field called Causal Discovery (figuring out cause-and-effect relationships in data, like "Does smoking cause cancer?").

Think of this like a cooking competition:

The Ingredients: 13 different datasets (some famous, some made up).
The Tools: 4 different cooking methods (algorithms) to find the recipe.
The Judges: The researchers ran each tool 100 times on each dataset to get the real truth (the ground truth). They knew exactly how well each tool worked.

Then, they asked 8 different top-tier AI models (like the smartest versions of ChatGPT, Claude, etc.) to look at the ingredients and the tool, and guess the results before running the test. They asked the AI to give a range: "I think the success rate will be between X% and Y%."

The Results: A Total Disaster

The results were shocking. Here is what happened:

The "Magic 15%" Failure:
If you were to guess a number completely at random, you'd be right about 36% of the time (because the ranges the AI gave were huge). But the AI models were right only 15.9% of the time.
- Analogy: Imagine a weather forecaster who says, "There is a 50% chance of rain between 1 AM and 11 PM." Even with such a huge window, they are wrong 84% of the time.
Worse Than Random Guessing:
Seven out of the eight AI models performed worse than if you had just closed your eyes and picked a random number. They weren't just bad; they were confidently wrong.
The "Confidence" Trap:
The AI models gave very wide ranges (e.g., "Success rate: 0% to 100%"). You might think, "Oh, they are being humble and covering all bases!"
- The Twist: Even with these massive, overly cautious ranges, the actual result was still outside their prediction most of the time. They were like a dart player throwing darts blindfolded, but the bullseye was moving, and they missed it even when they threw the dart from across the room.

Why Did They Fail? (The "Memorization" Trap)

The researchers dug deeper to find out why the AI failed. They found that the AI wasn't actually "thinking" or "reasoning." It was memorizing.

The "Famous Student" Analogy:
Imagine a student who has memorized the answer key for a specific set of practice exams (the "Benchmark" datasets). If you ask them about those specific exams, they get a few right because they remember the answers.
But if you give them a new exam with the same rules but different numbers (the "Synthetic" datasets), they panic. They have no idea how to solve the problem; they only know the answers to the old questions.
The Evidence:
- When the AI was tested on famous, well-known datasets, it did slightly better (because it had "read" about them in its training data).
- When tested on brand-new, made-up data, its performance collapsed.
- The AI models agreed with each other on the famous tests (because they all memorized the same answers) but completely disagreed on the new tests (because they were just guessing).

The Conclusion: Don't Trust the AI to Pick Your Tools

The paper concludes that you cannot use current AI models to select the best algorithm for a job.

If you are a data scientist trying to decide which tool to use for a new project, asking an AI "Which algorithm will work best?" is like asking a person who has read every car manual to guess how fast a specific car will go on a specific track without ever seeing the car or the track. They might recite the specs, but they can't predict the outcome.

The Takeaway:
LLMs are amazing at describing what algorithms are (declarative knowledge), but they are "blind" to how those algorithms actually behave in the real world (procedural prediction). Until this changes, we cannot rely on them to make critical decisions about which computational tools to use. We still need to run the tests ourselves.

1. Problem Statement

The paper investigates a critical gap in the capabilities of Large Language Models (LLMs): their ability to reason about computational processes and predict algorithmic performance. While LLMs possess broad declarative knowledge, it is unclear if they can provide calibrated quantitative predictions regarding how well a specific algorithm will perform on a given problem instance.

The authors argue that practitioners often rely on LLMs for "zero-shot" algorithm selection or uncertainty estimation. However, if LLMs merely pattern-match training text rather than performing principled reasoning, their recommendations could be dangerously misleading. The paper defines this failure mode as "Algorithmic Blindness": the inability of LLMs to form calibrated expectations about algorithm performance based solely on problem structure and algorithmic descriptions.

2. Methodology

The study uses Causal Discovery as a rigorous testbed because it offers diverse algorithms, standardized metrics, well-defined ground truth, and prominent benchmark datasets likely present in LLM training corpora.

Experimental Setup

Models: Eight frontier LLMs were evaluated (e.g., Claude-Opus-4.6, GPT-5.2, DeepSeek-V3.2, LLaMA-3.3, etc.).
Algorithms: Four distinct causal discovery algorithms were tested:
- PC and FCI: Constraint-based methods.
- LiNGAM: Functional causal model (linear non-Gaussian).
- NOTEARS: Continuous optimization approach.
Datasets:
- 13 Datasets: 9 standard benchmarks from the bnlearn repository (e.g., Sachs, Asia, Alarm) and 4 synthetic datasets generated from random Directed Acyclic Graphs (DAGs) with varying node counts (12 to 60 nodes).
Ground Truth Generation:
- For each of the 52 experimental conditions (13 datasets $\times$ 4 algorithms), each algorithm was executed 100 times with bootstrap resampling.
- This generated 5,200 total executions to compute empirical means and 95% confidence intervals for four metrics: Precision, Recall, F1, and Structural Hamming Distance (SHD).
LLM Evaluation Protocol:
- LLMs were prompted with dataset characteristics and algorithm assumptions to predict ranges (intervals) for the four metrics.
- Three distinct prompt formulations were used to test robustness.
- Metric: Calibrated Coverage was defined as the fraction of cases where the LLM's predicted range contained the true algorithmic mean. A well-calibrated model should achieve coverage matching its stated confidence (e.g., 95% coverage for a 95% interval).

Baselines

Random Baseline: Uniformly random intervals within valid metric domains.
Heuristic Baseline: Conservative ranges based on prior literature statistics.

3. Key Contributions

Rigorous Testbed: Established causal discovery as a principled environment for evaluating LLM algorithmic reasoning, moving beyond surface-level pattern matching.
Large-Scale Calibration Study: Conducted the first study of this magnitude, spanning 8 models, 13 datasets, 4 algorithms, and 5,200 ground-truth executions.
Definition of "Algorithmic Blindness": Formalized the specific failure mode where LLMs possess declarative knowledge but fail to translate it into calibrated procedural predictions.
Evidence of Memorization: Provided behavioral evidence (via synthetic data degradation and cross-model agreement) suggesting that marginal LLM success is driven by benchmark memorization rather than generalization.

4. Key Results

Systematic Failure (Algorithmic Blindness)

Low Coverage: Across all 1,664 comparisons, the frontier LLMs achieved a mean calibrated coverage of only 15.9%. This means the true algorithmic mean fell outside the LLM's predicted range in 84.1% of cases.
Worse than Random: Seven of the eight models performed below the random baseline (36.5%). Even the best-performing model (Claude) achieved only 39.4%, a marginal improvement that is statistically indistinguishable from chance in practical terms.
Overconfidence & Miscalibration: LLMs produced prediction ranges 8 to 27 times wider than the true 95% confidence intervals, yet these wide ranges still failed to capture the true mean. This indicates miscalibrated beliefs rather than conservative uncertainty.

Memorization vs. Reasoning

The authors identified three signals indicating that LLM performance relies on memorized statistics rather than reasoning:

Algorithm $\times$ Metric Dissociation: Performance varied wildly based on the specific algorithm-metric pair. For example, NOTEARS showed high SHD coverage (41.3%) while PC showed near-zero (0.0%), despite no inherent algorithmic reason for such a disparity in difficulty. This suggests retrieval of specific benchmark papers reporting SHD values for NOTEARS.
Benchmark vs. Synthetic Degradation:
- Coverage on benchmark datasets (17.7%) was significantly higher than on synthetic datasets (11.7%).
- Monotonic Collapse: On synthetic datasets, coverage dropped monotonically as network size increased (from 20.3% at 12 nodes to 6.2% at 60 nodes).
- LiNGAM Anomaly: LiNGAM, which has the most extensive benchmark literature, showed a 23.2% drop in coverage on synthetic data, whereas newer algorithms like NOTEARS showed stability. This inverse correlation with "algorithmic difficulty" and direct correlation with "training data density" confirms memorization.
Cross-Model Agreement: Models converged on similar predictions for well-known benchmarks (low pairwise distance) but diverged wildly on synthetic data (high pairwise distance). This suggests they are retrieving shared training data for benchmarks but guessing independently when that data is absent.

Prompt Sensitivity

The results were robust to prompt variations. High Coefficient of Variation (CV%) in predictions across different prompt formulations indicated fundamental instability in the models' "beliefs" about algorithm performance, rather than a lack of capability unlocked by better prompting.

5. Significance and Implications

Negative Finding for Zero-Shot Selection: The paper concludes that LLMs should not be used as zero-shot performance predictors for algorithm selection in causal discovery. Their predictions lack the calibrated validity required for decision-making.
Declarative vs. Procedural Gap: The study highlights a fundamental disconnect: LLMs can correctly describe algorithmic assumptions (declarative knowledge) but cannot apply them to predict empirical outcomes (procedural reasoning).
Limitations of RAG: The authors argue that standard interventions like Retrieval-Augmented Generation (RAG) would address factual gaps but cannot solve the absence of a structured "performance model" linking algorithmic assumptions to dataset properties.
Future Directions: The paper suggests that future work should explore LLMs as feature extractors for learned statistical predictors rather than direct predictors, and calls for human expert benchmarks to determine if this is a fundamental limitation of the task or a specific LLM failure.

In summary, the paper provides strong empirical evidence that current frontier LLMs suffer from algorithmic blindness, failing to generalize algorithmic reasoning beyond the specific statistics memorized from their training corpora.