Technical Summary: KG-HTC

Problem Definition

Hierarchical Text Classification (HTC) involves assigning documents to labels organized within a multi-level taxonomy, a task critical for domains like e-commerce categorization and literature management. While supervised methods have dominated previous research, they face significant barriers in real-world industrial applications:

1. Data Scarcity: Manually annotating data across multiple hierarchical levels is prohibitively expensive.
2. Large Label Spaces: Taxonomies often contain hundreds or thousands of leaf categories (e.g., the Amazon dataset has over 500 leaf categories).
3. Long-Tail Distributions: Datasets are often highly imbalanced, where a small fraction of frequent categories dominates while the majority of classes are underrepresented.

These factors render standard supervised approaches unsuitable for many industry settings, necessitating a shift toward strict zero-shot learning methods. Existing zero-shot approaches, such as those leveraging in-context learning (Halder et al.) or embedding-based similarity propagation (Z-STC, HiLA), often struggle with performance degradation as the hierarchy deepens, failing to effectively navigate large, sparse label spaces.

Methodology: KG-HTC

The authors propose KG-HTC, a novel framework that integrates Knowledge Graphs (KGs) with Large Language Models (LLMs) to address HTC challenges in a zero-shot setting. The method treats the label taxonomy as a Directed Acyclic Graph (DAG) and employs a Retrieval-Augmented Generation (RAG) pipeline to provide structured semantic context to the LLM.

The system architecture operates through the following stages:

1. Storage and Representation:

   • The label taxonomy is stored in both a Graph Database (Neo4j) to preserve hierarchical parent-child relationships and a Vector Database (ChromaDB) containing embeddings of all labels.
   • This dual storage allows the system to leverage topological constraints for semantic disambiguation.

2. Subgraph Retrieval (RAG):

   • Given an input text $x$, the system computes its embedding $\Psi(x)$.
   • For each hierarchical level $l$, the system calculates the cosine similarity between the input and all labels at that level.
   • Candidate labels $Q_l$ are retrieved based on a similarity threshold $\tau_l$.
   • Algorithm 1 validates cross-level dependencies: a candidate label at level $l$ is only retained if its parent label exists in the candidate set of level $l-1$. This process constructs a relevant subgraph (or sub-tree) from the full knowledge graph, filtering out irrelevant noise.

3. Prompt Construction:

   • The retrieved subgraph is transformed into a structured prompt using Algorithm 2.
   • The system performs an upward propagation traversal, enumerating all valid paths from leaf nodes to the root.
   • These paths are serialized as sequences (e.g., Level 1 -> Level 2 -> Level 3) and concatenated into a prompt. This chain-based representation ensures the LLM understands the hierarchical constraints without being overwhelmed by the entire taxonomy.

4. Layer-wise Classification:

   • The LLM performs classification iteratively from Level 1 to Level $L$.
   • For the first level, all top-level labels are included in the prompt.
   • For subsequent levels, the candidate set is constrained by the predicted label of the previous level ($y_{l-1}$) combined with newly retrieved candidates ($Q_l$). This strategy mitigates error propagation and handles large label spaces efficiently.

Key Contributions

The paper outlines three primary contributions:

1. Novel Integration: The introduction of KG-HTC, a method that integrates Knowledge Graphs into LLMs specifically for strict zero-shot Hierarchical Text Classification.
2. Semantic Retrieval Pipeline: The implementation of a pipeline that dynamically retrieves semantically relevant subgraphs based on cosine similarity and transforms them into structured, hierarchical prompts, enabling the LLM to reason over complex taxonomies.
3. Comprehensive Evaluation: An in-depth assessment demonstrating that KG-HTC outperforms previous state-of-the-art zero-shot methods (Z-STC and HiLA) across three open-source datasets, particularly in deeper hierarchy levels.

Experimental Results

The authors evaluated KG-HTC on three datasets: Amazon Product Reviews (3 levels), Web of Science (WoS) (2 levels), and DBpedia (3 levels). The primary metric was F1-macro, chosen to account for class imbalance.

• Performance Gains: KG-HTC significantly outperformed both a weak baseline (direct LLM classification) and strong baselines (Z-STC and HiLA).
  • On the Amazon dataset, KG-HTC achieved an F1-macro of 0.445 at Level 3, compared to 0.249 for HiLA and 0.173 for Z-STC.
  • On WoS, KG-HTC achieved 0.630 at Level 2, surpassing HiLA (0.371) and Z-STC (0.462).
• Decay Analysis: The paper introduces an "average decay rate" metric to measure performance loss as the taxonomy deepens. KG-HTC exhibited the lowest decay rate (e.g., 0.15 on Amazon vs. 0.43 for HiLA), indicating that the method maintains robustness even in deep, sparse hierarchies where other methods fail.
• Ablation Studies:
  • Subgraph Retrieval: Replacing the RAG-based subgraph retrieval with the full knowledge graph (Full-KG) resulted in lower performance at deeper levels, confirming that filtering relevant context is crucial for handling large label spaces.
  • Model Generalization: The method successfully improved the performance of an open-source LLM (Qwen2.5-8b) over its baseline, demonstrating generalizability beyond proprietary models like GPT-3.5-turbo.

Significance and Claims

The paper claims that KG-HTC effectively addresses the core challenges of HTC in industrial settings: the lack of annotated data, large label spaces, and long-tail distributions. By integrating structured knowledge (via subgraph retrieval) with the generative capabilities of LLMs, the method enables accurate zero-shot classification without requiring fine-tuning.

The authors emphasize that their approach allows LLMs to "understand label semantics at various hierarchy levels" by providing explicit topological constraints. The results suggest that incorporating structured knowledge is a viable path to overcoming the limitations of current zero-shot HTC methods, particularly in scenarios where label distributions are highly imbalanced and taxonomies are deep.

Limitations: The authors note that the method assumes the availability of a complete and correct label taxonomy; inaccuracies in the underlying knowledge graph would degrade performance. Future work is proposed to focus on using LLMs to construct more robust knowledge graphs.