Agentopic: A Generative AI Agent Workflow for Explainable Topic Modeling

1. Problem Statement

Traditional topic modeling techniques, such as Latent Dirichlet Allocation (LDA) and BERTopic, face significant limitations in real-world applications:

• Lack of Transparency: These models often operate as "black boxes," providing topic clusters without explaining why specific documents were assigned to them or how topics were grouped.
• Interpretability Gap: While statistical models (LDA) rely on word co-occurrence and embedding-based models (BERTopic) use vector clustering, neither offers natural language reasoning or traceable decision-making processes.
• Domain Constraints: In high-stakes fields like healthcare, finance, and public policy, opaque clustering can lead to misinformed decisions. There is a critical need for models that balance accuracy with Explainable AI (XAI) capabilities.
• Static Structures: Traditional models often produce flat, non-hierarchical topic structures that fail to capture the nuanced, multi-level semantic relationships within complex corpora.

2. Methodology: The Agentopic Workflow

The authors propose Agentopic, a novel multi-agent workflow that leverages Large Language Models (LLMs) to perform topic modeling through a coordinated, iterative process. The system is inspired by qualitative research thematic coding and Graph Neural Topic Models (GNTMs).

Core Architecture

Agentopic orchestrates five specialized agents in a structured pipeline:

1. Topic Identification Agent: Analyzes raw text (or user-seeded topics) to identify candidate topics. It generates natural language explanations justifying the identification.
2. Topic Review Agent: Validates the relevance and formatting of identified topics. If seeded topics are provided, it compares them against the seeds; otherwise, it flags errors for the Identification Agent. It embeds topic names and explanations into a Vector Database.
3. Topic Grouping Agent: Clusters validated topics into broader thematic groups based on semantic similarity and the generated explanations. It creates descriptions for each group.
4. Group Review Agent: Examines the structural validity of groups, checking for missing topics or structural flaws. It communicates with the Grouping Agent for refinements.
5. Hierarchy Construction Agent: Organizes the groups into a hierarchical tree (general-to-specific), synthesizing topic trees for easier interpretation. It feeds back into the Grouping Agent for iterative refinement.

Experimental Setup

• Dataset: The BBC News dataset (2,225 articles from 2004–2005) with five ground-truth categories: Business, Entertainment, Politics, Sport, and Tech.
• Preprocessing: A pipeline including contraction expansion, lowercasing, punctuation/digit removal, stopword removal, and lemmatization (applied to descriptions; titles were kept raw to preserve semantic value).
• Baselines:
  • LDA: Using TF-IDF vectorization.
  • BERTopic: Using all-MiniLM-L6-v2 embeddings and logistic regression.
  • GPT-4.1 Variants: Standalone models (nano, mini, full) prompted directly with text.
  • Agentopic: The same GPT-4.1 variants operating within the multi-agent workflow.
• Evaluation Metrics: Precision, Recall, and F1-Score across the five ground-truth categories.

3. Key Contributions

• Agentic Explainability: Unlike post-hoc explanation methods, Agentopic embeds explainability into the core workflow. Every step (identification, grouping, hierarchy) generates human-readable natural language justifications.
• Hierarchical Granularity: The system moves beyond flat categorization, generating a six-level hierarchical taxonomy with 2,045 semantically coherent topics, vastly expanding the original five-category structure.
• Traceable Decision-Making: The multi-agent design allows users to audit the logic behind topic assignments, enhancing trust and accountability.
• Dataset Augmentation: The workflow was used to augment the BBC dataset with generated explanations, enriching the dataset's context and utility for future research.

4. Results

The experiments demonstrated that Agentopic achieves high performance comparable to state-of-the-art models while offering superior interpretability.

• Quantitative Performance (F1-Score):
  • BERTopic: 0.98 (Highest, but black-box).
  • Agentopic (GPT-4.1 Full): 0.95 (Matches standalone GPT-4.1, outperforms LDA at 0.93).
  • Agentopic (GPT-4.1 Mini): 0.92 (Matches standalone Mini).
  • Observation: The agentic orchestration did not degrade performance despite the added complexity of reasoning and explanation generation.
• Qualitative Findings:
  • Coherence: Generated topics showed strong internal semantic consistency (e.g., grouping "training," "competition," and "outcomes" under Sports).
  • Coverage: The model uncovered fine-grained subtopics (e.g., separating "Domestic Affairs" from "International Relations" in Politics) that flat models missed.
  • Explainability: The system successfully generated clear rationales (e.g., linking "Time Warner profit" to "Economy and Finance" based on specific financial terms and regulatory context).

5. Significance and Implications

• Bridging the Gap: Agentopic successfully bridges the gap between statistical rigor, semantic richness, and human interpretability. It proves that high accuracy does not require sacrificing transparency.
• Domain Applicability: The framework is particularly valuable for high-stakes domains (finance, healthcare, law) where understanding the "why" behind a classification is as critical as the classification itself.
• Future of Topic Modeling: The paper suggests a paradigm shift from static, statistical clustering to dynamic, agentic workflows where AI agents collaboratively refine, validate, and explain data structures.
• Limitations & Future Work:
  • Current limitations include the lack of quantitative coherence validation (relying on the LLM's inherent reasoning) and potential redundancy in the hierarchy (e.g., nested "Sports" groups).
  • Future work aims to introduce human-in-the-loop corrections, automated pruning of redundant topics, and testing on multilingual and non-news datasets (e.g., social media, legal documents).

In conclusion, Agentopic represents a significant advancement in Explainable AI, offering a transparent, modular, and highly accurate alternative to traditional black-box topic modeling systems.