Domain Knowledge-Enhanced LLMs for Fraud and Concept Drift Detection

This paper proposes a Domain Knowledge-Enhanced LLM framework that integrates structured insights with a two-stage detection process to effectively identify deceptive conversations and classify concept drift as benign or fraudulent, achieving 98% accuracy and superior robustness compared to zero-shot baselines.

The Gist

The Big Problem: The "Shape-Shifting" Scammer

Imagine you are a security guard at a busy club (the internet). Your job is to spot fake guests (scammers) trying to sneak in.

In the past, scammers were easy to spot because they wore obvious disguises. But now, scammers are like shape-shifters. They start by talking normally about the weather or food (benign conversation), but then they suddenly shift gears to try to steal your credit card or trick you (fraud).

This sudden change in topic or tone is called "Concept Drift."

• The Problem: Sometimes, regular people change topics too (e.g., talking about the weather, then asking for a ride). A standard security guard might get confused, thinking a normal topic change is a scam, or worse, missing a scam because it started with a normal conversation.
• The Old Tools: Traditional computer programs are like guards who only memorize a list of "bad words." If a scammer uses new words or changes the topic, the guard misses them.
• The New Tool (LLMs): Large Language Models (LLMs) are like guards who can read and understand complex stories. However, they sometimes get confused, make things up (hallucinations), or don't know the specific rules of your club.

The Solution: The "Expert Guide" System

The authors of this paper built a three-part security team to catch these shape-shifting scammers. They didn't just give the AI a generic brain; they gave it a specialized instruction manual (Domain Knowledge) to help it understand the specific tricks scammers use.

Here is how their system works, step-by-step:

1. The First Guard: The "Fake Review" Detector

Before tackling complex conversations, they tested the system on fake reviews (like fake Yelp or Amazon reviews).

• The Analogy: Imagine a guard checking a guest list. Without help, the guard might think a very enthusiastic review is just a happy customer.
• The Upgrade: The team gave the guard a checklist of "suspicious signs" (e.g., "Is the praise too exaggerated?" "Does it sound like a robot?" "Are there weird buzzwords?").
• The Result: When the guard had this checklist, they got much better at spotting the fakes. For example, one AI model (Claude) went from being right 87% of the time to 95% just by using the checklist.

2. The Second Guard: The "Drift" Alarm (OCDD)

Once the system is watching a live conversation, it needs to know if the topic is changing.

• The Analogy: Imagine a conversation is a river. Usually, the water flows smoothly. Suddenly, the river hits a rock and changes direction.
• The Tool: They used a statistical tool called OCDD (One-Class Concept Drift Detector). This tool doesn't try to understand the meaning of the words yet; it just acts like a motion sensor. If the "flow" of the conversation changes too abruptly, the alarm goes off.

3. The Third Guard: The "Expert Interpreter"

When the alarm goes off, a second, smarter guard (a second LLM) steps in.

• The Job: This guard looks at the sudden change and asks: "Is this a harmless topic switch (like talking about the weather), or is this a trap (like a phishing attempt)?"
• The Secret Weapon: Just like the first guard, this one also has the specialized instruction manual. It knows that if someone suddenly asks for your credit card after talking about a job, that's a specific pattern of fraud.
• The Result: This system successfully told the difference between a harmless topic change and a malicious scam.

The Results: Who Won the Game?

The team tested this system using a dataset of real conversations (SEConvo) and compared it to older methods.

• The Champion: The LLaMA model (an open-source AI) was the star player. When given the "special instruction manual" (Domain Knowledge), it achieved 98% accuracy. It was far better than the older "team of guards" (traditional machine learning models) which only got about 82% right.
• The Lesson: Giving the AI specific knowledge about how scammers behave made it much smarter, more reliable, and easier to trust than just letting it guess on its own.

Summary

Think of this paper as a guide on how to train a security guard.

1. Don't just rely on memory: Old guards (traditional ML) forget new tricks.
2. Don't just rely on raw intelligence: Smart guards (LLMs) can get confused or make things up.
3. Give them a cheat sheet: By feeding the AI specific rules and patterns about fraud (Domain Knowledge), it becomes a super-guard that can spot the subtle, shape-shifting scammers that others miss.

The paper proves that when you combine a smart AI with a human's understanding of fraud tactics, you get a system that is highly accurate and can explain why it caught a scammer.

Technical Summary

Technical Summary: Domain Knowledge-Enhanced LLMs for Fraud and Concept Drift Detection

1. Problem Statement

Detecting deceptive conversations on dynamic digital platforms is increasingly challenging due to evolving language patterns and Concept Drift (CD). Concept drift refers to semantic or topical shifts that alter the context or intent of interactions over time. These shifts can obscure malicious intent or mimic normal dialogue, making accurate classification difficult.

While Large Language Models (LLMs) demonstrate strong performance in natural language tasks, they face significant limitations in high-stakes risk-sensitive scenarios:

• Contextual Ambiguity: Difficulty distinguishing between genuine topic shifts (benign drift) and manipulative pivots (adversarial drift).
• Hallucinations: A tendency to generate incorrect information when lacking specific domain grounding.
• Lack of Adaptation: Traditional Machine Learning (ML) models rely on fixed features and struggle with the fluid, ambiguous nature of multi-turn conversations.
• Interpretability: The "black box" nature of many models makes it difficult to understand why a conversation was flagged, a critical requirement for fraud detection.

The authors argue that fraud detection and concept drift should not be treated as isolated tasks, as fraudulent conversations frequently involve subtle shifts in tone or intent that resemble benign transitions.

2. Methodology

The paper proposes a Domain Knowledge (DK)-Enhanced Dual-LLM Framework designed to jointly model deception and drift. The methodology is executed in two phases:

Phase I: Fake Review Detection (Static Settings)

• Objective: To validate the value of domain knowledge in enhancing LLM performance for classifying static, single-turn deceptive content.
• Dataset: The Yelp Fake Review Dataset (20,000 reviews: 15,000 real, 5,000 fake).
• Approach:
  1. Pattern Discovery: An initial LLM analysis of labeled data identified specific linguistic markers of fake reviews (e.g., exaggerated praise, vague sentiment, unnatural phrasing, buzzword overuse).
  2. Structured Prompting: These identified patterns were encoded into structured prompts.
  3. Evaluation: LLMs (ChatGPT, DeepSeek, LLaMA, Claude) were tested in two conditions: (1) Zero-shot (no domain knowledge) and (2) Domain-Knowledge (DK) enhanced.

Phase II: Fraud and Drift Detection (Streaming Dialogues)

• Objective: To detect deceptive multi-turn conversations and classify the nature of semantic shifts.
• Dataset: The SEConvo dataset (400 multi-turn conversations: 40 for training, 360 for testing).
• Architecture: A three-component pipeline:
  1. DK-LLM Module 1: Detects whether a conversation contains fake or deceptive content.
  2. Drift Detection Unit (OCDD): A One-Class Concept Drift Detector trained on non-malicious conversations using a One-Class Support Vector Machine (SVM). It identifies statistical deviations (semantic shifts) in the dialogue stream.
  3. DK-LLM Module 2: If drift is detected by OCDD, this second LLM interprets the shift using domain-specific cues (e.g., requests for PII, urgency, excessive flattery) to classify the drift as either Benign (e.g., off-topic, spam) or Adversarial (e.g., phishing, manipulation).

3. Key Contributions

The paper outlines four primary contributions:

1. Domain Knowledge-Enhanced Dual-LLM Framework: A modular architecture that integrates structured domain insights into a dual-LLM system to simultaneously detect fraud, identify concept drift, and classify drift types—tasks often addressed in isolation.
2. Three-Stage Detection Pipeline: A structured workflow comprising a DK-LLM for initial deception flagging, an OCDD for semantic shift detection, and a second DK-LLM for drift classification. This enhances robustness and interpretability in streaming dialogues.
3. Impact of Structured Domain Prompts: Empirical evidence demonstrating that embedding domain-specific knowledge into prompt design significantly boosts performance and explainability across both synthetic (fake reviews) and real-world (SEConvo) datasets.
4. Comprehensive Benchmarking: An extensive evaluation of multiple LLM backbones (ChatGPT, Claude, LLaMA, DeepSeek) under zero-shot and domain-guided settings, identifying optimal configurations for fraud and drift detection.

4. Experimental Results

The study evaluated performance across multiple metrics (Accuracy, Precision, Recall, F1-Score) comparing models with and without Domain Knowledge (DK) and against traditional ensemble baselines.

• Phase I (Yelp Dataset):

  • Claude: Improved from 87% to 95% accuracy with DK.
  • DeepSeek: Showed substantial gains, rising from 56% to 90% accuracy.
  • LLaMA: Modest improvement from 52% to 58%.
  • ChatGPT: Marginal improvement (54% to 56%), suggesting sensitivity to prompt design variations.

• Phase II (SEConvo Dataset):

  • LLaMA emerged as the most robust model, achieving 98% across all metrics (Accuracy, Precision, Recall, F1-Score) with DK, a significant improvement over its 90% baseline.
  • DeepSeek improved its F1-score from 73% to 80%.
  • Comparison with Ensemble Models: The DK-enhanced Dual-LLM framework (LLaMA) significantly outperformed a traditional ensemble model (kNN, SVM, Imp-Kmeans) which achieved 82% accuracy. The LLM approach provided superior semantic depth and contextual reasoning.

5. Significance and Claims

The authors claim that their work demonstrates the viability of integrating domain knowledge into LLM workflows to achieve high precision and interpretability in complex, real-time detection scenarios.

• Enhanced Robustness: The framework effectively handles the interdependence between deception and drift, reducing false positives caused by benign topic shifts.
• Interpretability: By separating anomaly detection (OCDD) from semantic interpretation (LLM-2), the system provides clear reasoning for its classifications, as illustrated by sample outputs identifying specific phishing tactics.
• Model Sensitivity: The results highlight that not all LLMs respond equally to domain prompting; LLaMA and Claude showed the most significant gains, likely due to their training on dialogue-centric corpora and token-level reasoning capabilities.
• Limitations: The authors modestly acknowledge that LLMs still struggle with sarcasm, cultural idioms, and emotionally ambivalent language, which can lead to false positives. They also note the higher computational costs compared to traditional models, suggesting future work in prompt compression or hybrid lightweight architectures.

The paper concludes that while latency and cost are trade-offs, the gains in accuracy and interpretability make this approach suitable for high-stakes applications such as cybersecurity, online marketplaces, and customer service platforms.