Adversarial Training for Tabular Credit Scoring: A Multi-Attack Robustness Evaluation in P2P Lending

Technical Summary: Adversarial Training for Tabular Credit Scoring

Problem Statement
Machine learning-based credit scoring is increasingly central to Peer-to-Peer (P2P) lending, yet its resilience to adversarial manipulation remains poorly understood. In this context, applicants may strategically alter self-reported inputs to secure favorable lending decisions. While adversarial robustness is well-studied in image and text domains, evidence for tabular credit data is limited. Existing literature often evaluates a single attack against a matching defense, failing to address how defenses generalize across different attack types or how they perform under mixed threat scenarios. Furthermore, operational attackers are unlikely to adhere to a single threat model, suggesting that one-to-one attack-defense evaluations may overstate real-world resilience.

Methodology
The study employs a systematic train–test robustness benchmark using a large subset of the Lending Club dataset (nearly 400,000 observations). The experimental design follows a CRISP-ML inspired pipeline, focusing on evasion attacks at inference time.

• Models: Three distinct model families were evaluated: Logistic Regression (linear baseline), a Feed-Forward Neural Network (NN), and a Transformer-based model for tabular data (FT-Transformer).
• Adversarial Attacks: Four specific attack algorithms were implemented, constrained to applicant-mutable features (e.g., annual_inc, dti, revol_bal, revol_util, purpose, zip_code):
  1. Fast Gradient Sign Method (FGSM): A white-box, gradient-based attack.
  2. Projected Gradient Descent (PGD): An iterative white-box, gradient-based attack.
  3. DeepFool: A boundary-seeking attack that computes minimal perturbations to cross decision boundaries.
  4. Salt-and-Pepper (S&P) Noise: A non-gradient perturbation simulating data corruption or coarse manipulation.
  5. Mixed Regime: A training strategy combining multiple attack types.
• Experimental Design: The study utilized a full grid of training and testing configurations. Models were trained on clean data and on data augmented with specific attack types (FGSM, PGD, S&P, DeepFool, or Mixed). These models were then evaluated on clean test sets and test sets perturbed by each of the five attack types.
• Constraints: Perturbations were strictly limited to features an applicant could plausibly manipulate to ensure realistic attack scenarios. Gradient-based attacks used binary masks to prevent updates to immutable features.
• Metrics: Performance was measured using ROC AUC, accuracy, precision, recall, and F1-score, with a specific focus on class-0 metrics (default detection) due to the higher cost of misclassifying defaults. Stratified 5-fold cross-validation was used to ensure estimation stability.

Key Results

• Clean Baseline: All three model families achieved strong discrimination on clean data (ROC AUC 0.903–0.906), with the Transformer and Logistic Regression showing slightly higher accuracy (0.826) than the Neural Network (0.822).
• Vulnerability of Clean Models: When evaluated on perturbed test sets without defensive training, clean models exhibited moderate degradation. Gradient-based attacks (FGSM/PGD) and DeepFool reduced ROC AUC to approximately 0.85–0.87. S&P noise caused similar declines. Notably, targeted attacks (FGSM/PGD) systematically reduced the number of predicted defaults, shifting cases toward the "Fully Paid" class.
• Adversarial Training Efficacy:
  • Matched Attacks: Adversarial training sharply improved robustness when the training attack matched the test attack. For instance, Neural Networks trained on FGSM/PGD achieved ROC AUC scores of 0.952–0.959 on corresponding test sets, significantly outperforming clean-trained baselines.
  • Cross-Attack Generalization (Gradient): Robustness transferred well within the gradient-based family; models trained on FGSM performed nearly identically to those trained on PGD when tested on the other.
  • Cross-Attack Generalization (Non-Gradient): Transfer to non-gradient corruption (S&P) was weak. Models trained on gradient-based attacks did not significantly improve robustness against S&P noise, and vice versa.
  • Mixed Training: The mixed training regime (combining multiple attack types during training) delivered the most balanced robustness across heterogeneous attacks. It preserved clean-test performance while providing strong resilience against FGSM/PGD and improved stability under DeepFool and S&P compared to single-attack defenses.
• Clean Performance Trade-off: For higher-capacity models (Neural Network and Transformer), improved robustness did not require a material sacrifice in clean-data performance. In some cases, adversarially trained variants maintained or slightly exceeded clean benchmark accuracy.

Significance and Contributions
The paper makes three primary contributions to the literature on credit risk and adversarial machine learning:

1. Empirical Benchmark: It provides a comprehensive evaluation of AML in P2P credit risk assessment, testing multiple qualitatively different perturbation mechanisms (gradient-based and non-gradient) while constraining attacks to realistic, mutable applicant features.
2. Mixed Training Analysis: It introduces and assesses mixed training–testing strategies, demonstrating that single-attack defenses may overstate real-world resilience. The study shows that mixed training offers a more reliable default for scenarios where the attacker's strategy is uncertain or heterogeneous.
3. Practical Guidance: By benchmarking against real-world lending data, the study offers practical guidance for model governance. It suggests that while adversarial training enhances robustness against specific threats, a multi-attack stress-testing approach is necessary to ensure resilience against the diverse and evolving tactics of real-world adversaries.

The authors conclude that while adversarial training effectively hardens models against specific threats, relying on single-attack defenses is insufficient for operational credit scoring pipelines. Instead, they advocate for mixed training strategies and the integration of adversarial stress testing into model governance to ensure systems remain robust under realistic, heterogeneous threat scenarios.