💬 NLP

Summaries as Centroids for Interpretable and Scalable Text Clustering

The paper introduces k-NLPmeans and k-LLMmeans, two scalable text-clustering methods that improve interpretability and efficiency by periodically replacing numeric centroids with human-readable textual summaries.

Original authors: Jairo Diaz-Rodriguez

Published 2026-02-10

📖 3 min read☕ Coffee break read

CC BY 4.0

Original authors: Jairo Diaz-Rodriguez

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a librarian in a massive, chaotic library where books are flying around and landing in random piles. Your job is to organize them into neat categories (like "Cooking," "Space Travel," or "History").

Traditionally, computers do this using a method called k-means. Think of this like the librarian trying to find the "middle" of a pile by calculating the average weight and size of every book. It works, but at the end of the day, the librarian can only tell you: "Pile #1 is a collection of objects with an average weight of 2.4 lbs and a height of 8 inches."

That’s not very helpful! You still don't know what the pile is actually about.

This paper introduces a smarter way to organize that library called k-NLPmeans and k-LLMmeans.

The Big Idea: The "Summary-as-a-Label" Method

Instead of just calculating the "average weight" of a pile, the researchers suggest that every once in a while, the librarian should stop, read a few books from the pile, and write a one-sentence summary of what that pile is about.

Then, instead of using "average weight" to find the center of the pile, the librarian uses that written summary as the new anchor.

Here is the metaphor:
Imagine you are sorting a huge bag of mixed LEGO bricks.

The Old Way (Standard k-means): You group them by finding the "average color" and "average shape." You end up with a pile that is "sort of grey and sort of square."
The New Way (This Paper): You look at the pile and say, "This is a pile of tiny grey window pieces." That sentence is much more powerful than a mathematical average. It gives the pile an identity.

The Two Versions

The researchers created two "flavors" of this librarian:

The Budget Librarian (k-NLPmeans): This librarian is fast and doesn't cost much. They use old-school, "extractive" methods—basically, they just pick the most important sentences already written in the books and stick them together. It’s like making a collage from existing scraps.
The Genius Librarian (k-LLMmeans): This librarian uses a super-intelligent AI (like ChatGPT). They read the books and write a brand-new, beautiful summary from scratch. It’s much more accurate and "human-sounding," but it costs a little more "brain power" (and money) to run.

Why does this matter?

The paper proves three big things:

It’s more "Human-Readable": Instead of looking at a spreadsheet of numbers, you can look at your clusters and see: "Oh, these are all questions about how to activate a credit card." It makes the data interpretable.
It’s more Accurate: Because the "center" of the pile is now a meaningful sentence rather than a blurry mathematical average, the clusters stay much tighter and more logical.
It Scales: They even showed that this works for "streaming" data—like a constant flood of tweets or news articles—by using a "mini-batch" approach (sorting small handfuls of books at a time rather than trying to swallow the whole library at once).

In Short...

The paper moves text clustering from "Math-speak" (averaging numbers) to "Human-speak" (summarizing ideas). It turns a pile of data into a collection of clearly labeled topics.

Technical Summary: Summaries as Centroids for Interpretable and Scalable Text Clustering

Problem Statement

Standard text clustering pipelines typically follow a two-step process: embedding documents into high-dimensional vector spaces and then applying algorithms like k-means to group them. While effective, k-means relies on numerical averaging to update centroids. This approach has two major drawbacks:

Lack of Interpretability: Centroids are abstract vectors that offer no human-readable insight into the cluster's semantic content.
Semantic Drift: Purely mathematical averaging can "blur" the contextual nuances of the original text, potentially leading to clusters that are mathematically optimal but semantically incoherent.

Methodology

The authors propose a novel modification to the k-means algorithm: Summary-as-Centroid. Instead of updating centroids solely through numerical means, the algorithm periodically replaces the numeric centroid with a textual summary of the cluster. This summary is then re-embedded using the same encoder to serve as the new centroid for the next iteration.

The paper introduces two primary variants:

k-NLPmeans (LLM-optional/Low-cost): Uses classical, deterministic extractive summarization techniques (e.g., TextRank, Centroid-based, or LSA-style methods) to create prototypes. This version is fast, stable, and requires no Large Language Model (LLM) calls.
k-LLMmeans (High-performance): Uses an LLM to generate abstractive summaries. To ensure scalability, the authors use a fixed per-iteration budget (the number of LLM calls depends on the number of clusters $k$ , not the dataset size $n$ ) and a few-shot sampling strategy (using k-means++ to select representative documents for the prompt) to stay within context window limits.

Extensions:

Mini-batch Extension: The authors adapt the method to mini-batch k-means, allowing the algorithm to handle large-scale, streaming data (e.g., social media or news feeds) incrementally.

Key Contributions

Novel Algorithmic Framework: A hybrid approach that alternates between standard numerical updates and semantic summarization updates, preserving the k-means objective while enhancing semantic alignment.
Scalability & Efficiency: Unlike existing LLM-based clustering methods that scale poorly with dataset size, this method caps LLM usage, making it cost-effective.
Interpretability: By producing human-readable prototypes, the method allows practitioners to audit, debug, and track the evolution of topics (as demonstrated in the StackExchange case study).
New Benchmark: The release of a StackExchange-derived benchmark specifically designed for evaluating streaming text clustering.

Results

The authors conducted extensive experiments across four datasets (Bank77, CLINC, GoEmo, and MASSIVE) using various embedding models (e.g., S-BERT, OpenAI’s text-embedding-3-small).

Superior Accuracy: Both k-NLPmeans and k-LLMmeans consistently outperformed vanilla k-means and other classical baselines (k-medoids, GMM, Spectral clustering) in terms of Clustering Accuracy (ACC) and Normalized Mutual Information (NMI).
LLM Performance vs. Cost: k-LLMmeans approached the accuracy of state-of-the-art LLM-heavy clustering methods (like ClusterLLM or IDAS) but at a significantly lower cost and with fewer LLM calls.
Streaming Success: In sequential experiments on StackExchange data, the mini-batch variants outperformed standard mini-batch k-means, proving effective for real-time topic tracking.
Robustness: The method showed stability across different embedding models and was robust to variations in the number of clusters ( $k$ ).

Significance

This paper bridges the gap between efficient numerical optimization and semantic LLM reasoning. By treating summaries as centroids, the authors provide a way to make unsupervised clustering "auditable" for humans without sacrificing the mathematical rigor of the k-means framework. The modularity of the approach—allowing users to choose between zero-cost NLP methods or high-accuracy LLM methods—makes it highly practical for real-world industrial applications ranging from customer support ticket routing to real-time social media trend analysis.