💬 NLP

Agentic Clustering: Controllable Text Taxonomies via Multi-Agent Refinement

The paper introduces "Agentic Clustering," a novel framework that replaces fixed programmatic pipelines with a multi-agent system led by an orchestrator LLM to dynamically adapt text taxonomy discovery to specific corpus structures and user constraints, achieving state-of-the-art performance on seven benchmarks.

Original authors: Simon Löwe, Emily Silcock

Published 2026-06-02

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Simon Löwe, Emily Silcock

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a massive, messy pile of unsorted papers—thousands of customer emails, news articles, or scientific abstracts. Your goal is to sort them into neat, meaningful piles so you can understand what's going on. This is called text clustering.

For a long time, computers tried to do this by following a strict, pre-written recipe (like a rigid assembly line). They would chop the text into pieces, guess the categories, and then force every document into one of those boxes. The problem? The recipe was fixed. If the pile of papers changed (e.g., from short bank queries to long news stories), the old recipe often failed, creating messy or useless groups.

This paper introduces a smarter way: Agentic Clustering. Instead of a rigid assembly line, imagine a flexible construction crew led by a Project Manager.

The Cast of Characters (The Agents)

The system uses a "Project Manager" (an Orchestrator) who doesn't do the sorting itself. Instead, it looks at the messy pile and calls in specific specialists as needed:

The Proposer (The Brainstormer): This agent grabs a random handful of papers and says, "Hey, these look like they belong together." It creates a few draft lists of categories.
The Synthesizer (The Editor): This agent takes all the different draft lists from the Brainstormers and merges them into one master list. It decides, "Okay, Brainstormer A and Brainstormer B both mentioned 'Billing Issues,' so let's combine those into one official category."
The Auditor (The Quality Control Inspector): This agent takes new papers that no one has seen yet and tries to file them into the new master list. If the Auditor gets stuck or can't find a good home for a paper, it flags a problem.
The Investigator (The Detective): If the Auditor finds a problem (like "These two categories seem too similar"), the Detective digs deeper. It searches the pile for specific evidence to decide: "Should we merge these? Should we split this one? Or should we drop it?"
The Critic (The Judge): Once the list looks clean, the Critic reviews it against a strict checklist: Are the categories too broad? Are there gaps? Do the descriptions make sense? If the Critic is happy, the list is final.

How It Works (The Magic)

The key difference is that the Project Manager is in charge. It doesn't just run the steps in a fixed order.

If the list has too many categories, the Manager tells the Synthesizer to merge them.
If the categories are too vague, the Manager sends the Investigator to find more evidence.
If you tell the system, "I only want 20 categories," the Manager makes sure the team respects that limit.
If you say, "Sort these by emotion," the whole team shifts focus to find emotional themes.

This makes the system adaptable. It can handle a pile of short bank queries just as well as a pile of long news articles, because it changes its strategy based on what the data actually looks like, rather than forcing the data into a pre-set mold.

The Results

The authors tested this "construction crew" approach on seven different real-world datasets (like banking queries, Reddit comments, and news topics). They compared it against the best existing methods, including:

Old-school statistical models (like LDA).
Modern methods that use "dense embeddings" (mathematical maps of words).
Other recent AI methods that try to generate categories.

The Outcome: The Agentic Clustering method won on almost every test. In many cases, it was up to 32% better at grouping the text correctly than the previous best AI methods. It was particularly good at figuring out the right number of groups and handling different types of text without needing to be reprogrammed.

The Catch (Limitations)

The paper is honest about a few things:

Cost: Because this system uses powerful AI models to "think" through the sorting, it costs more money to run than simple math-based methods.
Language: It was only tested on English text.
Bias: Since the "agents" are AI models, they might inherit the biases of those models (e.g., how they define certain topics).
Reproducibility: The system relies on specific, expensive AI models that aren't open-source, making it hard for others to run the exact same experiment.

In a Nutshell

Think of old clustering methods as a stamp machine: you feed in text, and it stamps it with a pre-set label, even if the label doesn't fit perfectly.

This new method is like a team of expert librarians. They look at the books, discuss how to organize them, check if the system works on new books, and refine the shelves until everything fits perfectly. It's smarter, more flexible, and produces a much better library.

Technical Summary: Agentic Clustering via Multi-Agent Refinement

Problem Statement

Current text-clustering methods utilizing Large Language Models (LLMs) typically follow a fixed, programmatic pipeline. These approaches generally split the task into two stages: first, an LLM generates a taxonomy of cluster names from data samples; second, the entire corpus is assigned to these labels via classification. While recent iterations add refinement steps (merging, splitting, or revising clusters), the sequence of LLM calls, the rules for stopping, and the criteria for structural changes are hardcoded in advance.

This rigidity creates two primary limitations:

Poor Generalization: Pipelines tuned for specific dataset structures fail to generalize to corpora with different granularities or organizational patterns.
Lack of Controllability: These methods cannot easily incorporate user-supplied constraints, such as a target number of clusters ( $k$ -range) or a specific clustering intent (e.g., grouping by methodology vs. topic), without rewriting the underlying code.

The authors argue that clustering is a problem with no single "right" answer; the optimal number of clusters, level of abstraction, and handling of boundary cases depend entirely on the corpus and the user's intent. A fixed pipeline cannot adapt to these variables dynamically.

Methodology: The Agentic Framework

The paper proposes an agentic alternative where a central Orchestrator inspects the state of the discovery process at each step and dispatches one of five specialized agents. Rather than executing a pre-defined sequence, the pipeline is assembled on the fly based on the data's needs and user constraints.

Inputs

The system accepts:

An unlabelled text corpus.
A target $k$ -range $[k_{min}, k_{max}]$ .
An optional task description (free-text) defining the clustering lens (e.g., "cluster by issue type").

Agent Roles

Proposer: Samples disjoint batches of the corpus and drafts candidate cluster sets. Multiple proposers may run in parallel with different stylistic directives (e.g., fine-grained vs. broad) to surface structure at multiple granularities.
Synthesizer: Merges the independent drafts from proposers into a single proposed taxonomy. It resolves overlaps by inspecting underlying texts and ensures the final count adheres to the user's $k$ -range. It produces a taxonomy with names, descriptions, and confidence labels.
Auditor: Validates the proposed taxonomy against unseen texts (drawn from a disjoint sampling regime). It assigns texts to clusters with confidence scores and identifies "weak" clusters or unclustered patterns.
Investigator: Dispatched when the Auditor or Critic highlights specific issues (e.g., "should clusters 3 and 5 be merged?"). It performs targeted retrieval (TF-IDF or ID-based) to gather evidence and recommends structural changes (merge, split, rename, add, remove, or no change).
Critic: Evaluates the taxonomy against a six-axis checklist: Overlap, Gaps, Granularity, Boundaries, Descriptions, and $k$ -range compliance. It scores each axis as pass, warning, or failure.

The Loop

The Orchestrator manages the state. If the Critic finds issues, it dispatches the Investigator to resolve them. If the Investigator modifies the taxonomy, the Auditor re-runs to validate the changes on fresh evidence. This cycle continues until the Critic passes all six axes, at which point the taxonomy is declared final.

Final Classification

Once the taxonomy is finalized, the entire corpus is classified using a language model constrained to output only the valid cluster identifiers. This step uses a system prompt derived from the final taxonomy (without specific evidence examples) to ensure generalization.

Key Contributions

State-Dependent Orchestration: The paper introduces a framework where the clustering pipeline is not hardcoded but dynamically assembled by an orchestrator agent that adapts to the corpus structure and user constraints.
Specialized Agent Roles: The decomposition of the task into distinct roles (Proposer, Synthesizer, Auditor, Investigator, Critic) allows for targeted refinement and robustness against the limitations of single-pass generation.
Controllability: The method natively supports user-defined $k$ -ranges and task intents, forwarding these constraints to every agent to shape the discovery process.
Open Release: The authors release the approach as a "Claude Code skill" under the MIT License, providing the agent prompts and orchestration code.

Results

The method was evaluated on seven public text-clustering benchmarks (Banking77, CLINC150, MASSIVE-Intent, MASSIVE-Domain, GoEmotions, 20 Newsgroups, and StackExchange).

Performance: The agentic method achieved state-of-the-art performance on all datasets. It outperformed the strongest prior LLM baseline (LLM-embedding+k-means or similar) by up to 32% in Adjusted Rand Index (ARI).
Granularity Control: Unlike prior LLM-driven methods (e.g., TopicGPT, Huang & He) which struggled to control cluster granularity without gold-set leakage, the agentic approach successfully adhered to user-specified $k$ -ranges and intents.
Ablation Studies: Removing the user task description significantly degraded performance, confirming the importance of intent. Removing the post-synthesis refinement loop (Auditor/Critic/Investigator) had minimal impact, suggesting the core power lies in the sampling and synthesis phases.
Cost: While the method is more expensive than classical baselines (LDA, BERTopic) due to LLM usage, it is cost-competitive with other LLM-heavy methods. The authors note that the quality lead does not come at the highest price point compared to methods that scan the entire corpus multiple times with heavy models.

Significance and Claims

The paper claims that the agentic framing is significant because it treats clustering as a problem with multiple valid solutions dependent on context, rather than a fixed algorithmic output. By allowing the pipeline to "interrogate the corpus, observe what the data actually supports, and steer itself accordingly," the method achieves superior adaptability and performance.

The authors are modest regarding limitations:

Reproducibility: The method relies on frontier closed models (Claude Opus, GPT-5-mini), limiting reproducibility and locking users to specific vendors.
Language: Evaluation is restricted to English corpora; transferability to other languages is untested.
Risks: The authors flag risks regarding downstream decision-making (e.g., support ticket routing), the propagation of LLM biases into taxonomies, and the potential for the tool to be used to partition sensitive data along arbitrary axes (e.g., political affiliation).

The paper concludes that this approach represents a shift from rigid, programmatic pipelines to flexible, state-dependent agent systems for text clustering.