SCHK-HTC: Sibling Contrastive Learning with Hierarchical Knowledge-Aware Prompt Tuning for Hierarchical Text Classification

1. Problem Statement

Few-shot Hierarchical Text Classification (HTC) involves mapping text to a predefined tree-structured label hierarchy with extremely limited training samples. While existing methods utilize structural constraints to ensure parent-child prediction consistency, they face a critical bottleneck: distinguishing semantically similar "sibling" classes (labels at the same depth in the hierarchy).

Current approaches suffer from two main limitations:

• Insufficient Domain Knowledge: Standard Pre-trained Language Models (PLMs) lack the specific domain knowledge required to differentiate subtle semantic nuances between sibling classes, especially in data-scarce (few-shot) settings.
• Representation Overlap: Methods that pull lower-level label embeddings closer to enforce hierarchy often increase representational overlap, exacerbating confusion between fine-grained classes.
• Limitations of Existing Solutions:
  • HierVerb: Uses contrastive learning but struggles with deep-level ambiguity.
  • K-HTC: Incorporates Knowledge Graphs (KG) but lacks hierarchical awareness and effective fusion mechanisms.
  • DCL: Relies on Large Language Models (LLMs) and retrieval, which are computationally expensive and require extensive annotated data for in-context learning.

2. Methodology: SCHK-HTC Framework

The authors propose SCHK-HTC, a novel framework combining Prompt Tuning, Knowledge Graphs (KG), and Contrastive Learning. The architecture consists of two core components:

A. Hierarchical Knowledge-aware Encoder (HK-Encoder)

This module extracts structured, level-aware context from a Knowledge Graph (Wikidata) to compensate for data scarcity.

1. Entity Linking: Input text is processed to identify mentions and link them to entities in a KG subgraph (including one-hop neighbors).
2. Dual-Modality Encoding:
   • Semantic Modality: Entity embeddings are generated using BERT.
   • Structural Modality: Graph structure is encoded using Node2Vec and random neighbor sampling to capture topological relationships.
3. Fusion: Semantic and structural representations are fused via element-wise addition.
4. Prompt Integration: The model uses a cloze-style template (e.g., "the first layer's knowledge is [MASK]...") to inject this hierarchical knowledge into the PLM, extracting a knowledge-aware representation from the [MASK] token.

B. Sibling Contrastive Learning (SCL) Mechanism

To specifically address the ambiguity between sibling classes, the framework employs a dual-template contrastive learning strategy.

1. Hierarchical Context Encoder: Uses a prompt template like "the first layer is [MASK] rather than [MASK]..." where the positive [MASK] is the ground-truth label and the negative [MASK] is a confusable sibling label.
2. Hard Negative Mining: The model identifies "hard negatives" (labels with high predicted probability but incorrect) from the verbalizer output.
3. Verbalizer Contrastive Loss: The model is trained to maximize the similarity between the text representation and the ground-truth label embedding while minimizing similarity with hard-negative sibling embeddings. This forces the model to learn fine-grained distinctions.

C. Training Objectives

The total loss function is a weighted combination of four components:

1. Knowledge-aware Hierarchical InfoNCE Loss: Enforces structural constraints based on the label hierarchy (pulling samples with the same parent closer).
2. Sibling Contrastive Learning (SCL) Loss: Specifically targets the separation of sibling classes.
3. Verbalizer Classification Loss: Standard Cross-Entropy or Binary Cross-Entropy for the final classification task.
4. MLM Loss: Retains the Masked Language Modeling objective from the pre-trained BERT backbone.

3. Key Contributions

1. Novel Framework: Proposes SCHK-HTC, the first method to integrate hierarchical knowledge extraction from KGs with sibling-specific contrastive learning within a prompt-tuning paradigm for few-shot HTC.
2. Dual-Template Strategy: Introduces a mechanism to explicitly force the model to distinguish between semantically similar sibling classes using hard-negative mining and dual-template prompts.
3. Hierarchical Knowledge Fusion: Develops a method to extract and fuse both semantic and structural knowledge from KGs into the prompt-tuning process, providing crucial context for low-resource scenarios.
4. State-of-the-Art Performance: Demonstrates significant improvements over existing baselines, particularly in resolving deep-level classification ambiguity.

4. Experimental Results

The method was evaluated on three benchmark datasets: WOS (single-path), DBpedia (single-path), and RCV1-V2 (multi-path).

• Performance: SCHK-HTC consistently outperformed state-of-the-art baselines (including HierVerb, DCL, and HPT) across various shot settings (1-shot to 16-shot).
  • On WOS (16-shot), it achieved 81.31% Micro-F1 and 74.02% Macro-F1, surpassing the previous best (HierVerb) by a clear margin.
  • On DBpedia, it reached 96.26% Micro-F1 and 93.29% Macro-F1.
• Deep-Level Accuracy: As shown in Figure 1 of the paper, the model showed the most significant gains on the deepest layers of the hierarchy, where sibling confusion is highest.
• Ablation Studies: Removing the SCL loss caused the sharpest performance drop, validating its critical role in distinguishing sibling classes. Removing the HK-Encoder also degraded performance, confirming the value of external knowledge.
• Visualization: t-SNE plots demonstrated that SCHK-HTC creates clearer, non-overlapping clusters for sibling classes compared to HierVerb, which showed diffuse and overlapping distributions.

5. Significance

This paper addresses a fundamental limitation in few-shot hierarchical classification: the inability to distinguish fine-grained, semantically similar labels without abundant data. By effectively fusing external domain knowledge (via KGs) with contrastive learning specifically targeted at sibling relationships, SCHK-HTC provides a robust solution for real-world applications like news categorization and academic paper classification. It proves that prompt tuning, when augmented with structured knowledge and targeted contrastive objectives, can significantly outperform methods relying solely on text or massive parameter counts.