AI-Based Fraud Detection System for Subscription-Based Payment Services

Technical Summary: AI-Based Fraud Detection System for Subscription-Based Payment Services

Problem Statement
Subscription-based payment services (e.g., streaming, SaaS, recurring memberships) present a fraud landscape distinct from traditional one-off e-commerce transactions. While general payment fraud literature focuses on independent transactions characterized by amount, merchant, and location, subscription fraud exploits the recurring, temporal, and account-centric nature of billing cycles. Key challenges include:

• Recurring Structure: Fraudsters exploit predictable billing intervals and fixed amounts (e.g., card testing via small trial charges, repeated trial abuse).
• Account Lifecycle: Long-lived accounts accumulate behavioral histories that single-transaction models ignore, while legitimate self-service events (upgrades, pauses, device changes) create noise that mimics fraud.
• Delayed Ground Truth: Fraud confirmation (especially chargebacks) often occurs weeks or months after the transaction, leading to severe class imbalance and censored labels in training data.
• Gap in Literature: Existing AI research largely addresses general online payment fraud, leaving subscription-specific detection underexplored. Current systems risk missing subscription-specific patterns or over-flagging legitimate lifecycle events.

Methodology and System Architecture
The paper proposes a five-layer system architecture designed to handle real-time and batch scoring with a focus on auditability and graduated responses (allow, challenge, review, block):

1. Data Ingestion: Collects raw events from billing, authentication, and client systems.
2. Feature Engineering: Computes a ten-category feature taxonomy derived from the fraud lifecycle:
   • Static/Structured: Transaction amount, payment frequency, device info, IP/location, multi-account relationships.
   • Sequential/Temporal: Renewal patterns, user behavior sequences, failed payment attempts, subscription changes, and chargeback/refund behaviors.
3. Model Layer: Hosts comparative models and proposed hybrid architectures.
4. Decision Layer: Maps risk scores to actions based on configurable thresholds.
5. Feedback Layer: Captures delayed outcomes (fraud confirmation, chargebacks) to retrain models.

Model Comparison and Hybrid Architectures
The study evaluates a progressive chain of models: Logistic Regression, Random Forest, XGBoost/LightGBM, LSTM/Bi-LSTM, and Autoencoders. It identifies that tree-based models excel at structured features while sequence models capture temporal dependencies, but neither alone addresses the full problem. Consequently, two hybrid architectures are proposed:

• Hybrid I (XGBoost + Bi-LSTM): Combines an XGBoost classifier for structured features with a Bidirectional LSTM encoder for sequential billing history. This is designed for settings where labeled fraud data is available.
• Hybrid II (XGBoost + Autoencoder): Combines XGBoost with an unsupervised Autoencoder trained on legitimate billing patterns. The Autoencoder's reconstruction error serves as an anomaly signal, useful when labeled fraud is too scarce for supervised sequence training.

Evaluation Strategy
Recognizing the scarcity of real labeled data and the temporal nature of the problem, the paper employs a synthetic subscription-billing dataset generator and a time-based walk-forward validation protocol.

• Dataset: Simulates 2,600 accounts over 730 days with injected fraud types (trial abuse, card testing, account sharing, hijacking, chargeback, refund abuse) and realistic noise.
• Handling Imbalance: Uses SMOTE for structured features, class-weighted loss for sequence models, and censored-label handling (treating unconfirmed events as legitimate during training to mimic real-world delays).
• Metrics: Focuses on Precision, Recall, F1, and AUC-PR (preferred over AUC-ROC for imbalance), alongside business metrics like loss reduction and customer friction rates.

Key Results
Evaluated on 12 independently seeded synthetic datasets, the results demonstrate internal consistency with the design rationale:

• Hybrid I Performance: Significantly outperformed both its component models (XGBoost and Bi-LSTM) across all detection-quality metrics (Precision, Recall, F1, AUC-PR). It also Pareto-dominated plain XGBoost, achieving higher loss reduction (45.2% vs. 42.5%) with lower customer friction (2.2% vs. 4.5%).
• Hybrid II Performance: Significantly outperformed the standalone Autoencoder and XGBoost on Precision, F1, and AUC-PR. However, it showed a statistically significant, albeit small, deficit in Recall and Loss Reduction compared to XGBoost alone, trading these for substantially lower friction.
• Fraud-Type Specificity: The models achieved high recall (>90%) for "hijacking" and "refund abuse" (detectable at authorization time) but near-zero recall for "chargeback/friendly fraud," empirically confirming that chargeback fraud is a post-transaction risk estimation task rather than an authorization-time classification problem.
• Robustness: The qualitative ranking of models remained stable across parameter-perturbed generator variants (noisier signals, smaller fraud rings).
• Zero-Shot Generalization: When a specific fraud type (credential sharing) was removed from training, supervised models collapsed to zero recall, indicating they learn specific signatures rather than generalizable "sharing" concepts. The unsupervised Autoencoder retained minimal detection ability but did not significantly generalize.

Significance and Claims
The paper positions this work as a proof-of-concept that addresses a specific gap in the literature: the absence of a feature taxonomy, model-comparison methodology, and system architecture tailored to the recurring, account-centric structure of subscription billing.

• Contribution: It provides a structured approach to combining static and sequential features for subscription fraud, demonstrating that hybrid architectures can outperform component models on synthetic data designed specifically for this problem.
• Limitations: The author explicitly states that the results are not validated real-world performance. The findings are based on a synthetic construction and a single-machine evaluation. The specific numbers (e.g., 45.2% loss reduction) are conditional on the synthetic data's fraud-injection assumptions and the chosen decision thresholds, not on calibrated real-world cost optimization.
• Future Work: The paper identifies the need for validation on real, anonymized data, the extension to graph-based neural networks for relationship modeling, and the implementation of reinforcement learning for dynamic threshold adjustment.

In summary, the paper argues that while general payment fraud models are insufficient for subscription services, a hybrid approach leveraging both structured account features and sequential billing history offers a promising, theoretically sound direction for detection, provided it is evaluated with rigorous temporal validation and an understanding of delayed ground truth.