🤖 machine learning

Improving Detection of Rare Nodes in Hierarchical Multi-Label Learning

This paper proposes a weighted loss objective that combines node-wise imbalance weighting with focal weighting based on ensemble uncertainties to improve the detection of rare nodes in hierarchical multi-label classification, resulting in significant gains in recall and $F_{1}$ scores.

Original authors: Isaac Xu, Martin Gillis, Ayushi Sharma, Benjamin Misiuk, Craig J. Brown, Thomas Trappenberg

Published 2026-06-11

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Isaac Xu, Martin Gillis, Ayushi Sharma, Benjamin Misiuk, Craig J. Brown, Thomas Trappenberg

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Problem: The "Deep Forest" of Data

Imagine you are trying to organize a massive library. At the entrance, you have broad sections like "Science" or "History." As you walk deeper into the library, the shelves get more specific: "Biology," then "Marine Biology," then "Deep Sea Corals," and finally, a very rare, specific type of coral found only in one spot.

In machine learning, this structure is called a Hierarchy. The broad categories are the "parents," and the specific, detailed categories are the "children."

The problem the researchers identified is that AI models are terrible at finding the "children" at the very back of the library. Why?

Rarity: There are very few books on that specific coral.
The Parent Trap: The AI sees that "Science" and "Biology" are everywhere. It becomes lazy. It thinks, "I’ll just predict 'Biology' and call it a day." It ignores the rare, specific details because they are hard to find and don't appear often in the training data.

This is called the Rare Node Problem. The model stays at the surface level and fails to dig deep for the fine-grained details.

The Solution: Two Special "Spotlights"

The authors propose a new way to train the AI, using a special mathematical formula (a "loss function") that acts like two different spotlights to force the model to look harder at the rare items.

Spotlight 1: The "Fairness" Weight (Imbalance Weighting)

Normally, if an AI makes a mistake on a common topic (like "Biology"), it gets a small scolding. If it makes a mistake on a rare topic (like "Rare Coral"), it also gets a small scolding. This is unfair because the rare topic is harder to learn.

The researchers change the rules. They give the AI a much bigger penalty when it misses a rare node.

Analogy: Imagine a teacher who gives extra credit for identifying a rare bird, but only normal credit for spotting a pigeon. The student (the AI) is now motivated to look closely for the rare bird because the reward (or penalty for missing it) is higher.
The Twist: They don’t just look at the data points; they look at the nodes (the categories) themselves. This prevents the AI from getting confused by the fact that rare birds often appear in photos with common pigeons.

Spotlight 2: The "Confusion" Weight (Focal Weighting)

The second spotlight focuses on uncertainty. In machine learning, "uncertainty" means the AI is unsure about its answer.

The researchers use a technique called an Ensemble. Instead of using one AI brain, they use a team of 10 AI brains looking at the same image.

If all 10 brains agree, the AI is confident.
If the brains disagree, the AI is uncertain.

The new method tells the AI: "Pay extra attention to the nodes where you are confused."

Analogy: Imagine studying for a test. You don’t waste time reviewing the questions you already know perfectly. You spend all your energy on the questions that confuse you. This method forces the AI to focus its training energy on the parts of the hierarchy it doesn’t understand yet, which are usually the rare, deep-level categories.

How They Tested It

They tested this method on two types of data:

Gene Products: Complex biological data with thousands of categories (like a very deep library).
Underwater Images: Photos of sea life (echinoderms like starfish and sea urchins).

The Results

The results were quite impressive, especially for the rare items:

Recall Boost: "Recall" measures how many of the actual rare items the AI found. The new method improved recall by up to five times compared to standard methods. In other words, the AI found five times more of the rare, deep-level categories.
Better Accuracy: The overall score (F1 score) also improved significantly.
Helps with Bad Data: The method was especially helpful when the data was messy or when the AI’s "eyes" (the image encoder) weren't very good. It helped the AI dig deeper even when the starting conditions were poor.

Why This Matters (According to the Paper)

The paper notes that in fields like oceanography or medicine, the "rare" items are often the most important.

Oceanography: Detecting a rare species of coral can signal important environmental changes.
Medicine: Identifying a rare gene product can help diagnose a disease.

By helping the AI look deeper into the hierarchy, this method ensures that these critical, rare details aren't ignored just because they are uncommon.

Summary in One Sentence

The researchers created a training method that punishes the AI more heavily for missing rare categories and forces it to focus on the parts it is most confused about, resulting in an AI that is much better at identifying specific, hard-to-find details in complex data.

Technical Summary: Improving Detection of Rare Nodes in Hierarchical Multi-Label Learning

Problem Statement
The paper addresses the challenge of detecting rare, fine-grained classes (nodes) in Hierarchical Multi-Label (HML) classification tasks. In HML, data is structured as a directed acyclic graph (DAG) or tree, where child nodes are subsets of their parents. This structure naturally induces class imbalance, as parent nodes are typically more frequent than their descendants. Existing methods often fail to predict deeper, rarer nodes, which are crucial for detailed scientific insights (e.g., identifying rare species in seafloor classification or specific gene products in medical domains). The authors argue that standard observation-based resampling techniques are suboptimal because they risk overemphasizing common parent nodes when sampling for rare child nodes due to the conjoined nature of hierarchical annotations.

Methodology
The authors propose a weighted loss objective for neural networks that augments the standard hierarchical constraint loss (specifically the "max constraint loss" from the Coherent Hierarchical Multi-Label Classification Neural Network, C-HMCNN) with two distinct weighting components:

Node-wise Imbalance Weighting: Instead of weighting based on observation frequency, the method assigns weights based on node frequency. The weight for a node is inversely proportional to its occurrence frequency (including its descendants). To prevent common nodes from contributing negligible gradients, the weights are rescaled to enforce a zero lower bound and a minimum weight parameter ( $\tilde{w}_0$ ). Crucially, these weights are applied only to positive annotations (where the node is present), as weighting negative annotations did not improve performance.
Focal Weighting via Uncertainty: Inspired by focal loss, this component emphasizes nodes where the model is uncertain. The authors utilize an ensemble of models to quantify uncertainty, arguing that standard confidence metrics ( $p(y|x;\theta)$ ) are unreliable. They explore four uncertainty measures:
- Binary Bayesian Model Average (bBMA): Based on the variance of the ensemble’s mean predictions.
- Gated Margin Uncertainty (GMU): A signal-to-noise ratio metric that captures agreement and variance among ensemble members.
- Epistemic Uncertainty: Calculated using the expected Kullback-Leibler (KL) or Jensen-Shannon (JS) divergence between pairs of models in the ensemble.
The final loss function combines these terms: $L_{focal} = (\tilde{w}_0 + \tilde{W}(Y, f)) \cdot (U_0 + U(\Theta(X))^k) \cdot L_{MC}$ , where $L_{MC}$ is the max constraint loss. The uncertainty term is computed without gradients to avoid ensemble collapse.

Key Contributions

Imbalance Weighting: The paper provides evidence that node-wise weighting, independent of sample frequency, is more effective than observation-based resampling for improving hierarchical classification, particularly for rare nodes.
Focal Weighting: The authors demonstrate that incorporating modern uncertainty quantification (specifically bBMA and GMU) into a focal loss term significantly improves the detection of rare nodes by focusing training on uncertain predictions.

Results
The approach was evaluated on 16 gene product datasets (using Functional Catalogue and Gene Ontology hierarchies) and a benthic imagery dataset (BenthicNet-E).

Gene Product Data: The proposed method significantly outperformed baselines (no weighting, LPROS resampling, HROS-PD resampling). Specifically, recall for rare nodes improved by up to a factor of five compared to the control. The combination of imbalance weighting with bBMA or GMU focal weighting yielded the highest F1 scores. Epistemic uncertainty measures underperformed compared to bBMA and GMU, likely because aleatoric uncertainty dominates in well-trained models.
Vision Data (Echinoderms): The method showed significant advantages in challenging scenarios, such as when using suboptimal (noisy/untrained) encoders or limited training data. As the encoder quality improved or data availability increased, the relative benefit of the weighting methods diminished, supporting the hypothesis that the approach is most beneficial for difficult, imbalanced tasks.
Ablation Studies: The authors found that combining imbalance weighting with LPROS resampling further improved F1 scores on gene product datasets by synergistically boosting both precision and recall. However, dynamic schedulers attempting to transition from weighted to unweighted loss during training were largely unsuccessful in preserving precision.

Significance and Claims
The paper claims that its node-based weighting approach effectively addresses the persistent problem of rare node detection in HML, which is often neglected by existing methods. By decoupling node emphasis from observation distribution and leveraging ensemble uncertainty, the method enables models to make deeper, more comprehensive predictions. The authors emphasize that their approach is particularly valuable for scientific domains where fine-grained, rare classifications are critical, and where data may be limited or encoders suboptimal. The code is made available to facilitate reproducibility.