🤖 AI

Pre-trained Large Language Models Learn Hidden Markov Models In-context

This paper demonstrates that pre-trained large language models can effectively learn and predict sequences generated by Hidden Markov Models through in-context learning, achieving near-optimal accuracy on synthetic data and competitive performance on real-world animal decision-making tasks, thereby establishing ICL as a powerful tool for uncovering hidden structures in complex scientific data.

Original authors: Yijia Dai, Zhaolin Gao, Yahya Sattar, Sarah Dean, Jennifer J. Sun

Published 2026-04-27

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Yijia Dai, Zhaolin Gao, Yahya Sattar, Sarah Dean, Jennifer J. Sun

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The "Detective in the Library" Analogy

Imagine you are a detective walking into a massive, ancient library. You find a series of notes left behind by a mysterious person. These notes aren't just random scribbles; they follow a hidden pattern. For example, every time the person writes "Apple," they almost always write "Banana" next, but only if they previously wrote "Red."

The "Hidden Markov Model" (HMM) is like that secret code. It’s a system where you see the results (the words "Apple" and "Banana"), but you can't see the hidden rules (the "Red" rule or the person's mood) that caused them. Usually, to crack this code, you need a supercomputer and a PhD in mathematics to run complex, heavy-duty calculations.

This paper asks a surprising question: What if we just handed these notes to a highly well-read, incredibly smart librarian (a Large Language Model like GPT or Llama) and asked, "Based on these examples, what will the next note say?"

What the Researchers Found

The researchers discovered that these "Librarians" (LLMs) are much better at being detectives than we thought. They don't need to be "re-trained" or given a math textbook; they just need to look at a few examples in the prompt (this is called In-Context Learning), and they start to "feel" the hidden pattern.

Here is the breakdown of their discovery:

1. The Librarian is a Natural Pattern-Seeker

The researchers tested the LLMs on thousands of "fake" secret codes. They found that the LLMs weren't just guessing; they were actually getting as close to "perfect" as mathematically possible. It’s as if the Librarian looked at ten notes and said, "I see the pattern now. The next note will definitely be 'Banana'."

2. Some Codes are Harder than Others

Not every secret code is easy to crack. The researchers found two things that make a pattern "blurry":

The "Chaos" Factor (Entropy): If the person writing the notes is very random and unpredictable, even the smartest Librarian will struggle.
The "Memory" Factor (Mixing Rate): If the hidden rules change very slowly or stay stuck in one mode for a long time, the Librarian needs to read a much longer "book" of notes before they can figure out what's going on.

3. Real-World Magic: Watching Animals Think

This wasn't just math games. The researchers applied this to real science. They gave the LLMs sequences of decisions made by mice and rats in lab experiments.

In one case, the LLM was actually better at predicting what the mouse would do next than the specialized mathematical models designed by human experts!
It turns out that animal brains follow these "hidden patterns," and LLMs are surprisingly good at "reading" the rhythm of a living creature's behavior.

Why This Matters (The "So What?")

In the past, if a scientist wanted to understand a complex system—like how a climate pattern shifts or how a brain processes a reward—they had to build a custom, expensive mathematical engine.

This paper suggests a new, faster way: The "Diagnostic Tool" approach.

Instead of building a complex machine, a scientist can simply "show" their data to an LLM.

If the LLM predicts the next step easily, the scientist knows, "Okay, my data has a clear, learnable structure."
If the LLM fails, the scientist knows, "My data is either too chaotic or the rules are changing too fast to be easily predicted."

In short: LLMs aren't just for writing poems or coding; they are becoming powerful, "plug-and-play" statistical detectives that can help scientists uncover the hidden rhythms of the natural world.

Technical Summary: Pre-trained Large Language Models Learn to Predict Hidden Markov Models In-context

Problem Statement

Hidden Markov Models (HMMs) are essential for modeling sequential data with latent (unobservable) structures, such as animal decision-making, ecological processes, and climate patterns. However, traditional methods for fitting HMMs—such as the Baum-Welch algorithm (an Expectation-Maximization approach) or Gibbs Sampling—are computationally intensive, prone to local optima (non-convexity), and require significant domain expertise.

While Large Language Models (LLMs) have demonstrated remarkable In-Context Learning (ICL) capabilities for fully observed Markov processes, it has remained an open question whether they can implicitly learn the more complex dynamics of HMMs, which involve latent states, transition dependencies, and observation emissions, without explicit parameter updates.

Methodology

The researchers conducted a multi-stage study using synthetic and real-world datasets to evaluate the predictive power of LLMs via ICL.

Synthetic HMM Experiments: The authors generated a massive variety of HMMs (234 configurations) by systematically varying:
- Dimensions: State space ( $M$ ) and observation space ( $L$ ).
- Mixing Rate ( $\lambda_2$ ): The second-largest eigenvalue of the transition matrix, controlling how fast the system reaches a stationary distribution.
- Entropy: The randomness of transition ( $A$ ) and emission ( $B$ ) matrices.
- Stationary Distribution Skewness: Whether certain states are more frequent than others.
Evaluation Framework: LLMs (specifically the Qwen and Llama families) were prompted with observation sequences. Their performance was measured by their ability to predict the next observation ( $o_{t+1}$ $o_{t + 1}$ ) and compared against:
- Theoretical Optimum: The Viterbi algorithm (using ground-truth parameters).
- Statistical Baselines: Baum-Welch, Spectral learning, $n$ -gram models, LSTMs, and Transformers.
Theoretical Analysis: The authors proposed a conjecture linking LLM ICL behavior to Spectral Learning algorithms, suggesting that LLMs perform a form of "improper learning" where they estimate observation operators to predict future sequences.
Real-World Validation: The methodology was applied to biological datasets from the International Brain Laboratory (IBL) (mouse decision-making) and Miller et al. (rat reward-learning) to test generalizability to complex, non-synthetic data.

Key Contributions

Demonstration of Near-Optimal ICL: Proved that LLMs can achieve prediction accuracy approaching the theoretical Viterbi optimum for HMMs.
Characterization of Scaling Laws: Identified how HMM properties (mixing rate and entropy) dictate the "learning curve" of an LLM.
Theoretical Conjecture: Provided a mathematical framework connecting ICL to spectral learning, explaining why LLMs scale with context length in a manner similar to statistical algorithms.
Practical Diagnostic Tool: Established a framework for scientists to use LLM ICL as a "litmus test" to assess the learnability and complexity of their own experimental data.

Key Results

Convergence to Optimum: In low-entropy and fast-mixing HMMs, LLM accuracy consistently converges to the Viterbi limit.
Impact of HMM Properties:
- Entropy: Higher entropy (more randomness) increases the gap between LLM performance and the optimum.
- Mixing Rate: Slower mixing (high $\lambda_2$ ) delays the convergence of the LLM, requiring much longer context windows to "understand" the latent structure.
Superiority over Baselines: LLM ICL outperformed the Baum-Welch algorithm and traditional neural networks (LSTM/Transformer) in stability and convergence speed across many settings.
Real-World Performance:
- On mouse decision-making data, LLM ICL (86.2% accuracy) actually surpassed the expert-designed GLM-HMM model.
- On rat reward-learning data, LLM performance was lower, which the authors correctly attributed to the higher entropy and non-stationary nature of the task.

Significance

This work shifts the perception of LLMs from mere "text generators" to powerful, zero-shot statistical engines. For the scientific community, it provides a highly accessible, computationally efficient alternative to traditional modeling. Instead of spending weeks tuning complex EM algorithms, a scientist can format their data as a text prompt and use an LLM to immediately diagnose whether their data contains a learnable underlying structure or is dominated by stochastic noise. This establishes a new frontier for using foundation models in automated scientific discovery and complex system analysis.