Detection, Attribution, Narration: An End-to-End Pipeline for Explainable Money Mule Identification

Technical Summary: Detection, Attribution, Narration: An End-to-End Pipeline for Explainable Money Mule Identification

Problem Statement
Money mule accounts are critical facilitators of financial fraud, yet detecting them at scale remains challenging due to the heterogeneous nature of transactional and behavioral data. Traditional detection in financial institutions relies on deterministic, threshold-based rules (e.g., transaction velocity limits). While interpretable, these rule-based systems suffer from a rigid precision-recall trade-off: tightening rules reduces false positives but allows sophisticated mules to evade detection, while loosening them overwhelms analyst queues. Furthermore, rule-based systems often operate on individual features in isolation, failing to capture the multi-dimensional, subtle anomalies characteristic of money mule behavior. A critical operational gap exists: detection alone is insufficient; every flagged customer requires human review. Consequently, the utility of a detection system depends not only on its statistical precision but on how quickly and confidently analysts can triage alerts, a requirement often underweighted in academic literature.

Methodology
The authors present an end-to-end pipeline comprising three sequential stages designed to address both detection and explainability:

1. Feature Engineering and Classification:

   • Data: The system ingests data from core banking transactions, onboarding records, device/channel metadata, and watchlist feeds.
   • Features: 280 engineered features are constructed at the customer level, organized into five groups: Transaction patterns, Account demographics, Network/graph topology, Temporal behavior, and Device/Session/Channel. These features are aggregated over multiple time windows (7, 30, 90, and 180 days) to capture both short-term bursts and long-term drift.
   • Model: A LightGBM classifier is trained for binary classification. To address class imbalance, negative sampling is used to achieve a 1:5 positive-to-negative ratio, with scale_pos_weight applied during training.
   • Calibration: Raw probability scores are adjusted using Platt scaling to ensure that predicted scores (e.g., 0.8) faithfully reflect empirical likelihoods, which is crucial for alert ranking.

2. Attribution Layer (TreeSHAP):

   • For customers exceeding the alert threshold, the model's predictions are decomposed using TreeSHAP.
   • To balance context with prompt conciseness, the top 10 features by absolute SHAP magnitude are selected for each prediction.
   • For each selected feature, the system extracts the feature name, raw value, SHAP contribution value, and direction of influence.

3. Narration Layer (LLM):

   • A self-hosted, open-weight Large Language Model (LLM) converts the structured SHAP attributions into natural-language narratives for analysts.
   • Deployment: Models are deployed on-premise (NVIDIA H100 GPU) to ensure data residency and compliance. Three models were evaluated: Gemma-3-27B, GPT-OSS-120B, and Qwen3-Next-80B-A3B.
   • Prompt Engineering: The prompt includes prediction context, the top-10 SHAP evidence tuples, and strict instructions to output a fixed JSON schema (red flags, reasons, evidence references). The LLM is constrained to cite only provided features and avoid speculation.
   • Selection: GPT-OSS-120B was selected for production due to its superior factual accuracy, typology plausibility, and inference speed (despite having the largest parameter count, its Mixture-of-Experts architecture and shorter response length resulted in lower latency).

Key Contributions

• High-Dimensional ML Classifier: A LightGBM model trained on 280 engineered features with calibration analysis demonstrating reliable probability estimates.
• LLM-Augmented Explainability: A novel pipeline that translates quantitative SHAP attributions into analyst-facing natural language narratives, bridging the "last-mile" gap between model output and operational action.
• Live Production Validation: The system was deployed in a live Anti-Money Laundering (AML) environment, providing real-world metrics on yield rates and alert volumes, which are rarely reported in academic literature.
• Operational Insights: Practical guidance on integrating ML and LLM components into regulated workflows, including strategies for mitigating LLM hallucinations and ensuring model governance.

Results

• Classification Performance: On a held-out test set, the model achieved a precision of 0.909 and a recall of 0.816 at an operational threshold of 0.8. The Expected Calibration Error (ECE) for the high-score region was 0.021.
• Production Yield Rate: In live deployment, the system achieved a yield rate of 89%, a significant improvement over the incumbent rule-based system's 61%. This indicates that 89% of alerts reviewed by analysts were confirmed as true money mules.
• Coverage and Volume: Monthly alert volume increased from 211 (rule-based) to 302 (ML pipeline). This expansion reflects broader true-positive coverage rather than increased noise, corresponding to a 60% incremental adverse detection beyond existing workflows.
• Feature Importance: Global SHAP analysis revealed that Account Demographics (e.g., customer age, account tenure) and Transaction Patterns (e.g., credit-to-debit ratios) were the dominant drivers. Notably, financially inactive accounts and specific demographic profiles were strong signals.
• LLM Evaluation: Among the tested models, GPT-OSS-120B scored highest on factual accuracy (4.20/5) and typology plausibility. Analyst feedback indicated that LLM-generated narratives significantly reduced cognitive load during triage compared to raw SHAP visualizations, allowing for faster hypothesis formation.

Significance and Claims
The paper claims that the primary bottleneck in AML detection is often not model accuracy but the usability of explanations. By integrating an LLM narration layer, the pipeline successfully translates complex feature attributions into actionable insights, directly driving analyst adoption. The authors assert that open-weight models deployed on-premise can satisfy strict data residency requirements without sacrificing explanation quality, making this approach viable for institutions unable to use external APIs. Furthermore, the study demonstrates that ML models can outperform rule-based systems not just in precision, but in scalability; the ML pipeline allows for increased alert volume with only marginal loss in yield rate, whereas rule-based systems struggle to scale without disproportionately inflating false positives. The work concludes that while residual risks regarding LLM hallucinations and label quality exist, structured prompt constraints and periodic human review offer a practical path to deploying these systems in high-stakes, regulated financial environments.