Joint Estimation of Credit Risk in Consumer Lending: A Multi-Gate Mixture-of-Experts Approach

Technical Summary: Joint Estimation of Credit Risk in Consumer Lending via Multi-Gate Mixture-of-Experts

Problem Statement

In consumer lending, credit risk management relies on calculating Expected Loss (EL), which is the product of the Probability of Default (PD) and Loss Given Default (LGD). Standard industry practice estimates PD and LGD sequentially using separate models (e.g., logistic regression, XGBoost, or LightGBM). This approach presents two primary limitations:

1. Underutilization of Shared Information: PD and LGD depend on overlapping borrower and loan characteristics (e.g., affordability, leverage, credit history), yet sequential modeling treats them as independent problems, failing to leverage shared representations.
2. Error Propagation: Because EL is the product of predicted PD and LGD, errors in either component model compound in the final EL estimate.
3. Misalignment of Objectives: Optimizing for individual component metrics (e.g., PD AUC or LGD MAE) does not guarantee optimal calibration of the combined expected loss, which is the critical input for regulatory capital and risk-based pricing.

Additionally, LGD estimation faces a specific challenge of right-censoring, where loans are still active or have incomplete recovery histories at the time of analysis, distorting the empirical distribution of loss severity.

Methodology

The study proposes a Multi-Gate Mixture-of-Experts (MMoE) framework to jointly estimate PD and LGD. This architecture is designed to enable selective parameter sharing, allowing tasks to utilize different combinations of shared "expert" networks.

Data and Preprocessing:

• Source: Public Lending Club loan data (origination years 2013–2016).
• Filtering: To address right-censoring, the sample was restricted to loans that reached contractual maturity by the observation date and had a terminal status of either "Fully Paid" or "Charged Off."
• Split: Chronological split into training (2013–2014), validation (2015), and test (Jan–Feb 2016) sets.
• Targets:
  • PD: Binary indicator (1 for Charged Off, 0 for Fully Paid).
  • LGD: Net economic loss rate for defaulted loans, calculated as $1 - \frac{\text{Total Recovered Principal} + \text{Recoveries} - \text{Collection Fees}}{\text{Funded Amount}}$. Non-defaulted loans are assigned a sentinel value and excluded from the LGD loss calculation.

Model Architecture:

• Experts: A shared pool of $K=4$ expert networks. Each expert is a Multi-Layer Perceptron (MLP) with two hidden layers (sizes 128 and 64), utilizing batch normalization, ReLU activation, and dropout (0.2).
• Gating Mechanisms: Two task-specific gating networks (one for PD, one for LGD) compute soft weights over the expert pool using a softmax function. This allows the model to route input features to different expert mixtures depending on the task.
• Heads: Dedicated prediction heads for PD (sigmoid activation) and LGD (bounded regression).
• Loss Function: The model optimizes a combined objective using homoscedastic uncertainty weighting (Kendall et al., 2018). This approach learns log-variance parameters ($s_{PD}, s_{LGD}$) to automatically balance the Binary Cross-Entropy loss for PD and the Mean Squared Error (MSE) for LGD (calculated only on defaulted observations).

Baselines:
The MMoE model is compared against:

1. Sequential XGBoost: Separate classifiers/regressors for PD and LGD.
2. Sequential LightGBM: Separate classifiers/regressors for PD and LGD.
3. Hard-Sharing Multitask Learning (MTL): A single shared MLP trunk with separate output heads, representing full parameter sharing without selective routing.

Evaluation Metrics:

• Component Metrics: AUC-ROC for PD; Mean Absolute Error (MAE) for LGD.
• Primary Metric: EL-Cal RMSE (Expected Loss Calibration Root Mean Squared Error). This measures the agreement between predicted and realized EL rates across decile buckets of the portfolio.

Key Results

1. Superior Calibration of Expected Loss:
While sequential models and multitask models showed comparable performance on individual component metrics (PD AUC and LGD MAE), the joint estimation approaches significantly outperformed sequential baselines in EL calibration.

• Sequential XGBoost and LightGBM produced EL-Cal RMSE values of 0.1114 and 0.1016, respectively.
• The Hard-Sharing MTL model reduced this to 0.0222.
• The MMoE model achieved the lowest EL-Cal RMSE of 0.0202, representing a 9.0% improvement over the hard-sharing benchmark.

2. Segment-Specific Performance:
The benefits of MMoE were not uniform across the credit risk spectrum:

• Low Risk (Grades A–B): Hard-sharing MTL slightly outperformed MMoE (0.0098 vs. 0.0103 EL-Cal RMSE), likely because loss rates are low and homogeneous in this segment, offering little scope for task-specific routing.
• Middle Risk (Grades C–D): MMoE reduced EL-Cal RMSE by approximately 18.3% compared to hard sharing.
• High Risk (Grades E–G): MMoE reduced EL-Cal RMSE by approximately 3.5% compared to hard sharing.

3. Gate Weight Diagnostics:
Analysis of the learned gating weights revealed that the model utilizes selective routing effectively:

• Task Differentiation: The PD gate primarily routes based on a "credit quality" dimension (e.g., FICO score, interest rate), while the LGD gate incorporates variables more specific to loss severity (e.g., exposure size, revolving balances, employment history).
• Risk Segmentation: As credit quality deteriorates (moving from Grade A to E–G), the routing patterns shift systematically. For instance, the PD gate shifts weight toward experts associated with credit stress, while the LGD gate reallocates weight from experts associated with larger balances to those associated with higher risk profiles. This confirms that the model learns distinct expert mixtures for different tasks and borrower segments.

4. Robustness:
Sensitivity analysis on the number of experts ($K \in \{2, 4, 6, 8\}$) showed that $K=4$ yielded the best performance. Increasing capacity to $K=8$ degraded performance, suggesting that excess capacity can destabilize the routing mechanism.

Significance and Claims

The paper claims three primary contributions to the literature and practice of credit risk modeling:

1. Shift in Evaluation Focus: The study emphasizes that the ultimate goal of credit risk modeling is the calibration of the combined expected loss, not just the accuracy of individual components. It demonstrates that models with similar PD AUC and LGD MAE can produce vastly different expected loss estimates, arguing that component-level metrics are insufficient for assessing operational risk models.
2. Adaptation of MMoE to Credit Risk: It successfully adapts the Multi-Gate Mixture-of-Experts architecture to the joint estimation of PD and LGD. The results validate that selective parameter sharing (via task-specific gates) provides additional value over both sequential estimation and hard-sharing multitask learning, particularly in middle-to-high risk segments where the relationship between default occurrence and loss severity is more complex.
3. Empirical Evidence of Selective Routing: The study provides portfolio-level and segment-level evidence that selective routing improves calibration. The analysis of gate weights offers descriptive proof that the model learns to utilize different combinations of shared representations for PD and LGD, and that these routing strategies vary systematically across credit-risk segments.

The authors conclude that while joint estimation accounts for most of the improvement in expected loss calibration, the MMoE architecture's ability to route tasks to specific expert mixtures provides a further, statistically significant gain, making it a flexible framework for modeling related but distinct credit-risk outcomes.