💬 NLP

Revisiting Real-Time Digging-In Effects: No Evidence from NP/Z Garden-Paths

Through two experiments comparing human behavior with large language models on NP/Z garden-path sentences, this study finds no evidence for real-time digging-in effects, suggesting that previously observed trends are likely artifacts of sentence-final wrap-up processes rather than genuine strengthening of structural commitments during online processing.

Original authors: Amani Maina-Kilaas, Roger Levy

Published 2026-03-26

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Amani Maina-Kilaas, Roger Levy

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are walking down a path in a forest. The path splits, and for a moment, you aren't sure which way to go. You pick a direction and start walking. Suddenly, you hit a dead end. You have to stop, turn around, and retrace your steps to find the right way. In the world of reading, this is called a "garden-path" sentence.

For example: "Before the recruiters could adapt the policy changed."
Your brain initially thinks "the policy" is the thing being adapted. But then you hit the word "changed," and your brain realizes, "Oh no, 'the policy' is actually the subject of a new sentence!" You have to backtrack. This causes a momentary stumble, or a "reading slowdown."

The Big Question: Does "Digging In" Happen?

For decades, scientists have debated what happens before you hit that dead end.

Theory A: The "Self-Reinforcing" Model (Digging In)
Imagine you are walking down that wrong path. The longer you walk down it, the more you convince yourself, "No, this must be the right way." You get deeper into the wrong turn. By the time you hit the dead end, you are so committed to the wrong path that it takes a massive effort to turn around.

The Prediction: If you make the wrong path longer (by adding extra words like "for hiring"), you should get more stuck. The longer the ambiguity, the harder the recovery. This is called a "digging-in effect."

Theory B: The "Statistical Guessing" Model (Surprisal)
Imagine your brain is a super-smart GPS. It doesn't "commit" to a path; it just constantly calculates probabilities. "Based on the words I've seen, there's a 90% chance the next word is X, and a 10% chance it's Y."

The Prediction: If you add extra words to the wrong path, your GPS doesn't get "stuck." It just updates its math. Unless those extra words actually change the statistical odds of what comes next, the difficulty of turning around should stay the same. In fact, some modern AI models suggest that longer ambiguous paths might actually make the turn easier because the extra words give the brain more context to realize the mistake sooner.

The Experiment: A New Look at the Old Path

The authors of this paper wanted to settle this debate. They ran two experiments using human readers and compared them to a team of 16 different Large Language Models (LLMs)—essentially, very advanced AI that predicts the next word based on statistics.

They used two methods:

The Maze Task: A game where you have to choose the correct word to continue a sentence from two options. This is like a "spotlight" that measures exactly where you stumble, word-by-word, with very little distraction.
Self-Paced Reading: The classic method where you press a button to reveal one word at a time.

The Twist: They tested sentences where the "dead end" happened in the middle of the sentence versus at the very end.

The Results: What Did They Find?

Here is the surprising discovery, broken down simply:

1. The "Middle of the Sentence" Test (The Clean Data)
When the confusion happened in the middle of the sentence (where the brain is just processing the flow), humans did NOT show any "digging-in" effect.

What happened: If the wrong path was longer, humans didn't get stuck harder. In fact, they showed a tiny trend in the opposite direction (getting slightly easier to recover), which matched the predictions of the AI models.
The Metaphor: It's like walking down a long, confusing hallway. The longer the hallway, the more clues you see along the way that tell you, "Wait, this isn't the right way!" so you are actually ready to turn around sooner, not later.

2. The "End of the Sentence" Test (The Messy Data)
When the confusion happened at the very last word of the sentence, humans DID show a "digging-in" effect. The longer the wrong path, the harder it was to finish.

Why? The authors argue this isn't because the brain got "stuck" in the middle. It's because of "Wrap-Up."
The Metaphor: Imagine you are walking a path and you reach the finish line. Even if you were confused earlier, your brain suddenly tries to "pack up" the whole story, summarize it, and close the book. If the story was long and confusing, this "packing up" process is harder. The difficulty wasn't about being stuck during the walk; it was about the struggle to finish the walk.

The AI Connection

The AI models (LLMs) predicted the human behavior perfectly for the "middle of the sentence" tests: they predicted no digging-in, and humans agreed. However, the AI models failed to predict the "end of the sentence" difficulty, because AI doesn't have a human brain's "wrap-up" process.

The Bottom Line

The old idea that "the longer you are wrong, the harder it is to fix it" (Digging In) does not hold up when we look at real-time reading in the middle of a sentence.

Old View: The brain gets stubborn and digs its heels in.
New View: The brain is a statistical machine. It doesn't get stubborn; it just updates its probabilities. The "stubbornness" people saw in previous studies was likely an illusion caused by the brain struggling to wrap up a long, confusing sentence at the very end.

In short: Our brains are better at math than we thought, and they don't get "stuck" just because a sentence is long. They only get "stuck" when they are trying to tie a bow on a very messy package at the end.

1. Problem Statement

The paper investigates the validity of "digging-in effects" in human sentence processing.

The Phenomenon: A digging-in effect occurs when the difficulty of reanalyzing a sentence (disambiguation) increases as the length of the ambiguous region increases. For example, in an NP/Z garden-path sentence (e.g., "Before the recruiters could adapt the policy..."), if the ambiguous noun phrase is extended (e.g., "...adapt the policy for hiring..."), the reader is hypothesized to have "dug in" deeper to the incorrect syntactic analysis, making the subsequent reanalysis harder.
Theoretical Conflict:
- Self-Organizing/Dynamical Models: Predict digging-in effects. They posit that syntactic commitments strengthen over time via self-reinforcement; the longer a parse is maintained, the harder it is to revise.
- Surprisal Theory: Predicts no digging-in effect unless the added length genuinely shifts the statistical probability of upcoming words. If the lengthening does not change the expectation, difficulty should remain constant or potentially decrease (reverse digging-in) if the longer context provides more statistical evidence against the incorrect parse.
The Gap: Previous empirical support for digging-in (e.g., Tabor & Hutchins, 2004) relied heavily on sentence-final disambiguation. This confounds real-time processing with "wrap-up" processes (post-sentence integration), leaving the status of digging-in as a robust real-time phenomenon unclear.

2. Methodology

The authors conducted two human experiments and compared the results against predictions from an ensemble of Large Language Models (LLMs) operationalizing Surprisal Theory.

Experimental Design

Stimuli: English NP/Z garden-path sentences.
- Ambiguity: Ambiguous (garden-path) vs. Unambiguous (resolved early via comma or object).
- Length: Short (ambiguous region) vs. Long (ambiguous region + post-nominal modifier, e.g., "for hiring").
- Finality: Sentence-final disambiguation vs. Sentence-nonfinal (medial) disambiguation.
- Resolution: Comma vs. Object disambiguation.
Experiment 1: Maze Task
- Method: Participants choose between two words at each position to continue a sentence.
- Advantage: Provides highly localized, word-by-word processing measurements with minimal spillover effects.
- Participants: 167 valid participants.
- Design: 30 critical items; 15 per participant (Latin square).
Experiment 2: Self-Paced Reading (SPR)
- Method: Participants press a button to reveal words one by one (non-cumulative).
- Advantage: Replicates the methodology of the original Tabor & Hutchins (2004) study.
- Participants: 225 valid participants.
- Design: 36 critical items with within-item manipulation of all factors (Ambiguity, Length, Resolution, Finality).

Computational Modeling (Surprisal Predictions)

Model Ensemble: 16 LLMs from 5 families (GPT-2, Pythia, Gemma-3, Qwen 2.5, Mistral).
Prediction Strategy:
- LLMs were used to calculate surprisal (negative log probability) for words in context.
- A linear regression was fitted to human filler item reading times using surprisal, log-frequency, and word length as predictors.
- This model was applied to critical items to generate predicted reading times (RTs) for humans under Surprisal Theory.
- Individual LLMs were treated as "participants" in the statistical analysis to aggregate predictions.

Statistical Analysis

Models: Bayesian mixed-effects models (brms in R).
Coding: Sum coding for factors (+1/-1) to estimate deviations from the grand mean.
Significance: Defined by Probability of Direction ( $p_d$ ) > 97.5% (conceptually equivalent to $p < 0.05$ ).

3. Key Results

Human Behavioral Data

Garden-Path Effect: Robustly observed in both experiments (longer RTs at the disambiguating word for ambiguous sentences).
Digging-In Effect (The Core Finding):
- Sentence-Final Conditions: Showed a positive digging-in trend (longer ambiguous regions led to harder disambiguation). However, this is likely confounded by sentence wrap-up processes.
- Sentence-Nonfinal Conditions (Clean Test): Showed no evidence of digging-in. Instead, a reverse trend was observed (longer ambiguous regions led to slightly easier or neutral disambiguation), consistent with Surprisal Theory and neural network predictions.
Resolution Type:
- In the Maze task, comma-disambiguation produced significantly smaller garden-path effects than object-disambiguation, suggesting task-specific noise. This effect did not replicate in SPR.

LLM Predictions

Garden-Path Effect: LLMs successfully predicted the existence of garden-path effects (higher surprisal at disambiguation).
Digging-In Effect: LLMs consistently predicted reverse digging-in (or no effect) across both final and nonfinal conditions.
- This aligns with the human data in nonfinal conditions but contradicts the human data in final conditions.
Predictive Adequacy:
- LLMs accurately predicted RTs for non-critical regions.
- LLMs underpredicted the difficulty of critical words in ambiguous conditions (a known limitation of current models in handling structural violations).
- Crucially, the magnitude of underprediction was not greater for long conditions than short conditions, supporting the lack of a digging-in effect.

4. Key Contributions

Refutation of Real-Time Digging-In: The study provides strong evidence that digging-in effects are not a robust real-time phenomenon in NP/Z garden-paths when sentence-final wrap-up effects are controlled for.
Methodological Insight: It demonstrates that sentence-finality is a critical confound. Previous findings of digging-in likely reflect post-sentence integration processes rather than incremental structural commitment strengthening during parsing.
Human-LLM Convergence: In nonfinal conditions, human behavior qualitatively aligns with the predictions of neural language models (Surprisal Theory), suggesting that human processing difficulty tracks statistical expectations rather than time-dependent self-reinforcement.
Task Comparison: The Maze task proved superior to Self-Paced Reading for isolating incremental processing effects and minimizing spillover, offering a cleaner test for comparing humans and LLMs.

5. Significance and Implications

Theoretical Impact: The findings challenge dynamical, self-organizing models of parsing that rely on time-dependent strengthening of commitments. Instead, they support rational models (like Surprisal Theory) where difficulty is driven by information content and statistical probability, not merely the duration of the ambiguous state.
Modeling Limitations: The study highlights that while LLMs capture general surprisal patterns, they systematically underpredict the cost of structural reanalysis (garden-paths). However, the pattern of this underprediction (lack of digging-in) matches human behavior in real-time contexts.
Future Directions: Researchers should treat sentence-final positions with caution in psycholinguistic experiments aiming to measure incremental processing. Future work should focus on understanding why sentence-final wrap-up interacts with ambiguity and length to produce the positive digging-in trend observed in prior literature.

Conclusion: The paper concludes that there is no evidence for real-time digging-in effects in English NP/Z garden-paths. The apparent effect in previous literature is likely an artifact of sentence wrap-up processes, and human sentence processing in real-time appears to follow statistical expectations rather than self-reinforcing structural commitments.