Technical Summary: A Comparative Study of CNN-LSTM and Graph Neural Networks for Credit Card Fraud Detection

1. Problem Statement

The rapid expansion of digital transactions has outpaced traditional rule-based fraud detection systems, which struggle to capture the nonlinear, temporal, and relational patterns inherent in modern financial data. While deep learning offers a solution, a significant gap exists in the literature regarding a fair, controlled comparison between sequential deep architectures (specifically CNN-LSTM) and graph-based relational models (Graph Neural Networks, or GNNs).

Furthermore, fraud detection faces the critical challenge of extreme class imbalance (often <0.2% fraud cases), where standard accuracy metrics are misleading. Existing studies often lack side-by-side comparisons using identical preprocessing, validation protocols, and fairness-aware optimization, making it difficult to determine which paradigm is superior for anonymized, feature-rich transaction data.

2. Methodology

The authors conducted a controlled, end-to-end comparative study using the public Kaggle "Credit Card Fraud Detection" dataset (284,807 transactions, 492 fraudulent). The study employed identical data splits (70% training, 15% validation, 15% testing), preprocessing, and evaluation metrics for both models.

Data Preprocessing & Imbalance Handling

• Normalization: Features were standardized using a Standard Scaler.
• Class Imbalance: A class-weighting approach was applied during training, assigning higher loss weights to the minority (fraud) class.
• Threshold Tuning: Decision thresholds were optimized on the validation set to maximize the F1-score, moving beyond the default 0.5 cutoff to balance precision and recall.

Model Architectures

1. CNN-LSTM (Sequential/Hybrid):

   • Structure: A 1D Convolutional Layer (64 filters) to extract local feature correlations, followed by Batch Normalization, a Bidirectional LSTM (128 units) to capture temporal dependencies, Global Average Pooling, Dropout (0.3), and a Sigmoid output.
   • Rationale: Designed to capture spatial feature interactions and sequential dynamics simultaneously without requiring explicit relational graphs.
   • Training: Adam optimizer, binary cross-entropy loss, 25 epochs.

2. Graph Neural Network (GNN):

   • Structure: A GraphSAGE framework. Due to the anonymized nature of the dataset (no explicit relational attributes), a synthetic pseudo-graph was constructed. Cardholders and merchants were inferred as nodes, and transactions as edges, using quantile discretization for merchant binning.
   • Rationale: Designed to model relational structures and detect collusive behavior (fraud rings).
   • Training: Binary cross-entropy with logits (weighted) and a Focal Loss variant. Two versions were tested: standard weighting and Focal Loss with clipped weights.

Evaluation Metrics

The study prioritized metrics sensitive to imbalance over raw accuracy:

• Precision, Recall, F1-Score: To evaluate fraud class identification.
• ROC-AUC & PR-AUC: To measure discrimination ability independent of thresholds, with PR-AUC specifically highlighted for imbalanced datasets.
• Latency: Average inference time per sample to assess real-time deployment feasibility.

3. Key Results

The experimental results revealed a stark performance disparity between the two architectures on this specific dataset:

• CNN-LSTM Performance:

  • After threshold tuning (optimized at 0.9995), the model achieved a Precision of 0.9138, Recall of 0.7162, and an F1-score of 0.8030.
  • It maintained a high ROC-AUC of 0.9795.
  • The baseline model (threshold 0.5) achieved a PR-AUC of 0.8124; however, Section 5.1 notes that after threshold calibration, the precision-recall area was 0.792.
  • The model demonstrated low latency, making it suitable for real-time inference.

• GNN Performance:

  • The GNN struggled significantly. Section 4.3 reports a PR-AUC of just 0.012 and a ROC-AUC of 0.711 for the baseline model (BCE with pos_weight), with a precision of 0.0045 and recall of 0.5135.
  • Table 2 lists the GNN (BCE) PR-AUC as 0.0021, highlighting a discrepancy between the text description and the tabulated results.
  • Even with Focal Loss, the model failed to distinguish fraud, often predicting all transactions as fraudulent (Zero Precision/Recall).
  • The authors attribute this failure to the lack of meaningful relational data in the anonymized dataset; the synthetic graph construction could not generate sufficient signal for the GNN to learn.

4. Key Contributions

• Controlled Comparative Framework: The paper provides a reproducible, fair comparison between CNN-LSTM and GNN using identical datasets, preprocessing, and hyperparameter tuning protocols, addressing a gap in existing literature.
• Empirical Evidence on Anonymized Data: The study demonstrates that while GNNs are theoretically powerful for relational fraud detection, they fail to generalize on heavily anonymized, feature-only datasets where explicit relational attributes (e.g., shared devices, merchant networks) are absent.
• Importance of Threshold Calibration: The research highlights that threshold tuning is critical for practical deployment, significantly improving precision without sacrificing recall in highly skewed domains.
• Metric Selection: The paper advocates for the use of PR-AUC and F1-score over accuracy to provide a realistic assessment of fraud detection capabilities.

5. Significance and Claims

The paper claims that for anonymized, feature-rich transaction data, CNN-LSTM architectures are superior to GNNs. The hybrid model successfully leverages temporal sequencing and feature interactions without requiring external graph structures.

The authors emphasize that data characteristics dictate model suitability:

• CNN-LSTM is the preferred choice for real-time, feature-based detection in environments where relational data is unavailable or anonymized.
• GNNs remain valuable but are contingent on the availability of explicit relational attributes (e.g., cross-merchant networks, device sharing) to function effectively.

The study concludes that future fraud detection systems may benefit from hybrid approaches that combine sequence modeling with graph reasoning, provided the necessary relational data is accessible. It also underscores the necessity of methodological rigor in AI research, ensuring that comparisons are not confounded by inconsistent preprocessing or evaluation metrics.