💬 NLP

Agent-Driven Corpus Linguistics: A Framework for Autonomous Linguistic Discovery

This paper introduces Agent-Driven Corpus Linguistics, a framework where an LLM autonomously conducts the full cycle of linguistic inquiry by generating hypotheses and querying corpora via a structured interface, thereby producing empirically grounded, falsifiable findings at machine speed while lowering technical barriers for researchers.

Original authors: Jia Yu, Weiwei Yu, Pengfei Xiao, Fukun Xing

Published 2026-04-09

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Jia Yu, Weiwei Yu, Pengfei Xiao, Fukun Xing

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a detective trying to solve a mystery about how people speak and write. In the old days, you had to do all the work yourself: you'd read thousands of books, highlight sentences, count words on your fingers, and try to guess patterns. It was slow, exhausting, and you could only check a few ideas before you ran out of time and energy.

This paper introduces a new partner for your detective work: Agent-Driven Corpus Linguistics.

Think of this not as a replacement for you, the detective, but as a super-powered AI intern who never sleeps, never gets tired, and can read a million books in the time it takes you to brew a cup of coffee.

Here is how this new system works, broken down into simple concepts:

1. The Setup: The Librarian and the Robot

The Human (You): You are the Chief Detective. You decide the direction of the investigation. You might say, "I want to know how English intensifiers (words like very, really, utterly) have changed over time."
The AI Agent: This is your Robot Intern. It has a direct line to a massive digital library (a "corpus") containing millions of words.
The Connection (MCP): Imagine a special walkie-talkie that lets the Robot talk to the library. The Robot doesn't just guess; it asks the library specific questions, gets the exact data, and reports back.

2. The Workflow: A Cycle of Discovery

Instead of you doing the heavy lifting, the Robot takes over the investigative loop:

Hypothesize (The Guess): You give the Robot a broad mission. The Robot looks at its training (what it already knows) and says, "I bet very used to be rare, but now it's everywhere. I also bet really is used more in plays than in poems."
Query (The Search): The Robot goes to the library and runs a search. It doesn't just read; it counts. It finds exactly how many times very appears in 18th-century books versus 20th-century books.
Interpret (The Analysis): The Robot looks at the numbers. "Oh! The data shows very peaked in the 1700s and then dropped. And really is indeed 20 times more common in drama than in poetry!"
Iterate (The "Wait, what?" Moment): This is the magic part. If the Robot finds something weird, it doesn't just stop. It says, "That's strange. Let me ask the library another question to see why." It keeps looping, refining its ideas, until it has a solid story.
Report (The Solution): Finally, the Robot hands you a report with charts, numbers, and a clear story. You, the Chief Detective, review it and say, "Yes, that makes sense. This is our finding."

3. Why is this a Big Deal?

The paper tested this Robot on a specific mystery: English Intensifiers (words like very, really, utterly).

The "Grounding" Advantage: If you just ask a normal AI (without the library connection) to guess, it might make things up or give vague answers like, "I think really is popular." But because this Robot is grounded in real data, it can say, "In dramatic plays, really appears 352 times per million words, but in poetry, only 17 times. That's a 20-fold difference." It turns vague guesses into hard facts.
The "New Discovery": The Robot didn't just confirm what humans already knew. It found a new pattern: it realized that some words (like utterly) didn't just lose their meaning; they became specialized for negative things (like utterly useless or utterly hopeless). It grouped these changes into three distinct "paths" of evolution, something a human might take years to figure out.

4. The "Replication" Test

To prove the Robot wasn't just hallucinating, the researchers asked it to solve two other famous mysteries that human experts had already solved.

Mystery A: How the word reader declined in popularity over 200 years.
Mystery B: How people started using "remembering doing" instead of "remembering to do."

The Robot solved both of these with numbers that were almost identical to the human experts' results. It proved that the Robot can be trusted to do the math and the searching.

5. The Bottom Line: A Partnership, Not a Replacement

The paper argues that this isn't about replacing human linguists.

Humans are the Architects: We have the intuition, the theory, and the final judgment. We know why a pattern matters.
The AI Agent is the Construction Crew: It does the boring, repetitive, heavy lifting. It checks every single brick, counts every nail, and builds the structure so fast that we can see the whole picture.

In short: This paper shows that we can now hire an AI to be our "research assistant" that reads the entire library, runs the experiments, and does the math, leaving us free to focus on the big ideas and the storytelling. It lowers the barrier to entry, meaning anyone with a question can now do high-level linguistic research without needing to be a coding expert.

1. Problem Statement

Corpus linguistics traditionally relies on human researchers to formulate hypotheses, construct complex queries (often in specialized languages like CQP), interpret results, and iterate on findings. This process faces two primary bottlenecks:

Technical Barrier: Corpus query languages require specialized training, limiting access for researchers with strong theoretical backgrounds but limited technical skills.
Time and Attention Constraints: The "observation–interpretation–correction" loop is slow when driven by a single human. Researchers are often forced to limit the number of hypotheses tested or cross-tabulations performed due to time limitations, potentially missing subtle patterns in large, richly annotated corpora.

While Large Language Models (LLMs) have been used as assistants for query translation or literature synthesis, they have not yet been deployed as autonomous research agents capable of driving the full investigative cycle (hypothesis generation $\to$ querying $\to$ interpretation $\to$ refinement) without constant human intervention.

2. Methodology: The Agent-Driven Framework

The authors propose Agent-Driven Corpus Linguistics, a framework where an AI agent, rather than a human, performs the cognitive work of corpus investigation.

A. Conceptual Positioning

The framework introduces a new dimension to the existing "corpus-based" vs. "corpus-driven" distinction:

Epistemological Axis: Concerns the relationship between theory and data (does theory precede data, or vice versa?).
Autonomy Axis: Concerns who performs the inquiry (human vs. AI).
The agent-driven approach is orthogonal to the epistemological axis; an AI agent can operate in corpus-based, corpus-driven, or hybrid modes. The core innovation is that the agent autonomously navigates the relationship between theory and data.

B. System Architecture

The system consists of three layers connected via the Model Context Protocol (MCP):

AI Agent Layer: An LLM (specifically Anthropic's Claude Opus) operating in a coding environment. It receives a natural language research directive, generates hypotheses, constructs queries, interprets results, and decides on the next steps.
MCP Server Layer: A lightweight Python server that exposes three specific tools to the agent:
- corpus_info(): Retrieves metadata (field mappings, syntax).
- cqp_query(): Executes queries and returns concordance lines (KWIC).
- cqp_frequency(): Returns quantitative frequency counts, optionally broken down by metadata (genre, period, etc.).
- Significance: This ensures every step is logged, reproducible, and grounded in actual tool calls rather than hallucinated statistics.
Corpus Engine Layer: The IMS Open Corpus Workbench (CQP/CWB) hosting the corpus data.

C. Operational Workflow

The research process follows a five-stage iterative loop:

Direction (Human): The researcher provides a broad research goal (e.g., "Investigate English intensifiers").
Hypothesize (Agent): The agent generates testable hypotheses based on parametric knowledge.
Query (Agent): The agent constructs and executes CQP queries via the MCP tools.
Interpret (Agent): The agent analyzes results, refines hypotheses, or discards them based on evidence. If new patterns emerge, it loops back to Stage 2.
Review (Human): The agent produces a structured report; the human researcher evaluates the final findings and decides on the research agenda.

3. Key Contributions

Methodological Definition: Formalizes "Agent-Driven Corpus Linguistics" as a complementary dimension to existing methodologies, defining three core properties: Grounding (claims anchored in verifiable data), Iterative Autonomy (self-directed refinement), and Expanded Analytical Coverage (machine-speed cross-tabulation).
Technical Implementation: Demonstrates a working system linking an LLM to a CQP-indexed corpus via MCP, proving the feasibility of protocol-agnostic, autonomous corpus inquiry.
Empirical Validation:
- Case Study: Autonomous discovery of diachronic relay chains, semantic change pathways, and register sensitivity in English intensifiers.
- Baseline Experiment: A controlled comparison showing that grounding in corpus data adds quantification, falsifiability, and data-responsive synthesis that the LLM cannot produce from training data alone.
- Replication: Successfully replicated two published studies (Claridge, 2025; De Smet, 2013) on an independent corpus (CLMET) with high quantitative accuracy.

4. Key Results

The framework was tested on a 5-million-token subset of the Project Gutenberg corpus focusing on English intensifiers.

Diachronic Relay: The agent identified a "relay chain" of intensifiers over time: so+ADJ (dominant in medieval/early modern) $\to$ very (peaked in 18th century) $\to$ really (rising in late 19th/20th century).
Semantic Change Pathways: The agent distinguished three distinct pathways of semantic change, moving beyond simple "delexicalization":
1. Complete Delexicalization: (e.g., very) becoming a pure degree marker with no semantic constraints.
2. Polarity Fixation: (e.g., utterly, truly) specializing in specific evaluative polarities (negative/privative or positive) rather than losing meaning entirely.
3. Metaphorical Constraint: (e.g., deeply) retaining source-domain constraints (spatial depth).
Register Sensitivity: The agent autonomously discovered a massive register gap for really (352 pmw in Drama vs. 17 pmw in Poetry), a 20-fold difference, identifying it as a colloquial marker.
Baseline Comparison:
- Without Corpus: The LLM could generate correct qualitative hypotheses and rank ordering but could not provide precise numbers, specific frequency counts, or falsifiable data points.
- With Corpus: The agent provided precise metrics (e.g., "20-fold difference"), revised hypotheses based on contradictory data, and synthesized a three-pathway framework based on empirical evidence.
Replication Success: The agent replicated the decline of the noun reader and the spread of gerund complements for the verb remember in the CLMET corpus, matching published frequencies and trends with high precision.

5. Significance and Implications

Lowering Technical Barriers: Researchers can now conduct rigorous corpus studies using natural language directives, bypassing the need to learn complex query languages like CQP.
Scalability and Coverage: Agents can perform systematic cross-tabulations across all metadata dimensions (genre, period, sentiment) at machine speed, uncovering patterns that human researchers might overlook due to time constraints.
Auditability: The framework generates a complete, reproducible log of every query and result, enhancing the transparency and reproducibility of corpus linguistics.
Epistemological Shift: The approach does not replace human expertise but redefines the division of labor: humans set the direction and evaluate the final output, while agents handle the labor-intensive cycle of hypothesis testing and data synthesis.
Future Potential: The framework is applicable to any structured data source accessible via tool-use interfaces, promising significant advancements in under-resourced languages and interdisciplinary digital humanities research.

Limitations Noted: The study acknowledges that the agent's initial hypotheses often rely on training data (recall) rather than pure discovery, and that the stochastic nature of LLMs can lead to variations in interpretive frameworks (though quantitative results remain stable). Future work requires larger, temporally balanced corpora and methods to further distinguish genuine discovery from training-data bias.