Technical Summary: Focused PU Learning from Imbalanced Data

1. Problem Statement

The paper addresses the challenge of Positive-Unlabeled (PU) learning in scenarios characterized by severe class imbalance and hard-to-detect positive examples. In many real-world applications (e.g., fraud detection, disease gene identification, financial misstatement detection), training data consists of a small set of labeled positive instances and a large pool of unlabeled instances. The unlabeled pool is dominated by negative instances but contains hidden positive ones.

Existing PU learning methods often rely on simplifying assumptions that fail in practical settings:

• Class Imbalance: Many methods assume balanced distributions or rely on sufficient labeled positives, which is rarely the case when the positive class is under-represented.
• Labeling Mechanisms: Most methods assume the Selected Completely At Random (SCAR) assumption, where labeled positives are a random subset of all positives. However, real-world data often follows the Selected At Random (SAR) assumption or the Probabilistic Gap (PG) assumption, where positive examples resembling negatives are less likely to be labeled.
• Hard Positives: When positive examples overlap significantly with the negative distribution (the PG scenario), standard risk estimators struggle to distinguish them, leading to poor performance.

The authors argue that current state-of-the-art methods (e.g., nnPU, PUHDT) either fail under SAR/PG assumptions or rely on data manipulation (resampling, two-step filtering) that may not be robust in highly imbalanced settings.

2. Methodology

The proposed method, termed iFPU (Imbalanced Focused PU), introduces a new non-negative risk estimator that integrates Focal Loss into the PU learning framework.

2.1 Theoretical Foundation

The approach builds upon the unbiased risk estimators for PU learning (uPU) and non-negative risk estimators (nnPU). In standard PU learning, the risk $R(g)$ is approximated using labeled positives ($P$) and unlabeled data ($U$) by replacing the negative risk term $R_N^-(g)$ with an expression involving $U$ and $P$:
$$\tilde{R}_{PU}(g) = \pi_P \tilde{R}_P^+(g) - \pi_P \tilde{R}_P^-(g) + \tilde{R}_U^-(g)$$
where $\pi_P$ is the class prior. To prevent negative empirical risks (which cause overfitting in neural networks), the nnPU method applies a non-negative constraint: $\max\{0, \tilde{R}_U^-(g) - \pi_P \tilde{R}_P^-(g)\}$.

2.2 Focused Risk Estimator

The authors observe that in imbalanced settings, the standard risk is dominated by the unlabeled data, causing the model to neglect the few hard-to-distinguish positive examples. To address this, they replace the standard cross-entropy loss with Focal Loss [50].

Focal Loss down-weights easy examples (those with high probability of correct classification) and up-weights hard examples. The focal loss for a probability $p_t$ is defined as:
$$FL(p_t) = -(1-p_t)^\gamma \log(p_t)$$
where $\gamma$ controls the focusing parameter.

The authors derive the iFPU risk estimator by substituting the standard loss terms in the PU risk equation with the focal loss formulation:
$$\tilde{R}_{iFPU}(g) = \frac{1}{n_P} \sum_{i=1}^{n_P} -\pi_P (1-g(x_i))^\gamma \log(g(x_i)) + \max\left\{0, \frac{1}{n_U} \sum_{i=1}^{n_U} -(g(x_i))^\gamma \log(1-g(x_i)) - \frac{1}{n_P} \sum_{i=1}^{n_P} -\pi_P (g(x_i))^\gamma \log(1-g(x_i)) \right\}$$

2.3 Training Procedure

The training process utilizes stochastic optimization (SGD). A key feature of the algorithm is the handling of the non-negative constraint:

• If the term $(\tilde{R}_U^-(g) - \pi_P \tilde{R}_P^-(g))$ is non-negative, the model updates parameters using the gradient of the full risk.
• If the term is negative (indicating potential overfitting or violation of the non-negative constraint), the model performs a gradient ascent step on the difference term to push it back toward non-negativity, rather than simply ignoring the gradient.

The method is designed to be end-to-end, compatible with any classifier based on cost minimization (e.g., MLPs, XGBoost), without requiring data preprocessing or resampling.

3. Key Contributions

1. Novel Risk Estimator: The derivation of a non-negative risk estimator specifically tailored for imbalanced PU data using Focal Loss, which explicitly targets hard-to-classify positive examples.
2. Relaxation of Assumptions: The method is evaluated under both SCAR and SAR assumptions, demonstrating robustness in scenarios where labeling bias exists (i.e., positives resembling negatives are less likely to be labeled).
3. End-to-End Integration: Unlike two-step methods that require identifying reliable negatives first, iFPU integrates directly into the training loop of standard classifiers (MLP and XGBoost).
4. Real-World Validation: The method is applied to a challenging real-world problem: financial misstatement detection, where PU data arises naturally due to delayed detection of errors.

4. Experimental Results

The authors evaluated iFPU on 14 publicly available imbalanced datasets and a real-world financial dataset.

4.1 Benchmark Datasets

• Setup: 14 datasets with varying imbalance ratios. Experiments were run under SCAR and SAR assumptions with 25%, 50%, and 75% of positive examples labeled.
• Comparators: Compared against uPU, nnPU, i-nnPU, NNIF (two-step), PUHDT, PUHRF, LBE, and SAREM.
• Performance:
  • Under SCAR, iFPU (using XGBoost) achieved results comparable to the state-of-the-art PUHRF (PU Hellinger Random Forest), with no statistically significant difference in most cases.
  • Under the more challenging SAR assumption, iFPU significantly outperformed most competitors. In the most difficult scenario (25% labeled positives), iFPU surpassed PUHRF by 5 percentage points in PR-AUC.
  • The method showed robustness to incorrect class prior estimates, maintaining performance even when the prior was significantly underestimated.

4.2 Real-World Application: Financial Misstatement Detection

• Context: Detecting financial misstatements in annual reports (2003–2014) where misstatements are rare and often undetected for years (SAR/PG scenario).
• Setup: Compared against supervised baselines (TabTransformer with calibrated focal loss, Fin-SVM, RUSBoost) and PU methods (PUHRF).
• Results:
  • The proposed iFPUXGB achieved the highest R-precision of 19.95%, outperforming the previous state-of-the-art TabTransformer model (17.63%) and PUHRF (18.22%).
  • The results indicate that modeling the problem as PU learning (acknowledging hidden positives) yields significantly better performance than treating unlabeled data as negative.

5. Significance and Claims

The paper claims that iFPU represents a state-of-the-art solution for PU learning in imbalanced settings, particularly where the Probabilistic Gap assumption holds.

• Robustness: The method's ability to perform well under SAR assumptions without requiring complex data manipulation or resampling is highlighted as a major advantage over existing tree-based or two-step methods.
• Practicality: The successful application to financial misstatement detection demonstrates the method's utility in high-stakes, real-world domains where labeled data is scarce and biased.
• Flexibility: By relying on a risk minimization framework compatible with various classifiers (MLP, XGBoost), the approach offers a flexible alternative to specialized PU algorithms.

The authors modestly note that while they avoided hyperparameter tuning to ensure fair comparison, further optimization of the XGBoost and focal loss parameters could potentially yield even stronger results. They also identify temperature scaling for calibration and extensions to multi-class settings as future research directions.