🤖 AI

Let Relations Speak: An End-to-End LLM-GNN Soft Prompt Framework for Fraud Detection

This paper proposes the LLM-GNN Soft Prompt Framework (LGSPF), an end-to-end approach that bridges graph structures and semantic space using soft prompts and a parallel GNN encoder to overcome text scarcity and feature distortion, thereby achieving state-of-the-art performance and enhanced interpretability in multi-relational fraud detection.

Original authors: Zhixing Zuo, Huilin He, Jiasheng Wu, Dawei Cheng

Published 2026-05-28

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Zhixing Zuo, Huilin He, Jiasheng Wu, Dawei Cheng

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Problem: The "Silent" Graph

Imagine you are a detective trying to catch a fraudster. Usually, you have a file full of notes, emails, and chat logs (text) that tell you what happened. But in the world of financial fraud, privacy laws often mean you don't have those notes. You only have a massive web of numbers: "Transaction A happened at 2:00 PM," "User B bought Item C," "They are 5 miles apart."

This is what the paper calls a Weak-TAG (Weak Text-Attributed Graph). It's a web of connections with no words attached.

The Dilemma:

Old AI (GNNs): These are like math wizards. They are great at looking at the web of numbers and finding patterns, but they are terrible at explaining why something is suspicious. They are "black boxes."
New AI (LLMs): These are like brilliant detectives who can read, reason, and explain their thoughts. But they are used to reading stories and sentences. If you feed them a wall of raw numbers, they get confused. They try to turn those numbers into words (like "The number 45.23..."), which breaks the meaning and makes them lose the plot.

The Solution: LGSPF (The "Silent Translator")

The authors propose a new system called LGSPF. Think of it as a universal translator that connects the "Math Wizard" (GNN) with the "Brilliant Detective" (LLM) without forcing the numbers to become clumsy words.

Here is how it works, step-by-step:

1. The Parallel Team (The GNN Encoder)

Imagine the fraud web has different types of connections: "Same Credit Card," "Same IP Address," and "Same Location."

Old way: You might mash all these connections into one big pile.
LGSPF way: It hires a team of specialized scouts (Parallel GNNs).
- Scout A only looks at "Same Credit Card" connections.
- Scout B only looks at "Same Location" connections.
- Scout C only looks at "Same IP" connections.
- Each scout creates a unique "report card" (an embedding) for the suspect based only on their specific type of connection. This ensures no detail is lost.

2. The "Soft" Handshake (Soft Prompt)

This is the paper's most creative trick.

The Hard Way (What others do): Trying to force the LLM to read a sentence like: "The user has 3 neighbors, 2 of which are frauds, and the distance is 5.4 miles." This is like trying to describe a painting by listing the hex codes of every pixel. It's messy and loses the "vibe."
The LGSPF Way (Soft Prompt): Instead of writing words, the system creates invisible, magical tokens (like secret handshakes).
- It says to the Detective: "Here is a secret token representing the 'Credit Card' pattern, and here is a secret token representing the 'Location' pattern."
- These tokens aren't words; they are direct, high-quality mathematical summaries of the patterns.
- The LLM doesn't have to "read" the numbers; it just "feels" the pattern through these tokens. It's like handing the detective a high-resolution photo of the evidence instead of a written description of the photo.

3. The Joint Training (End-to-End Optimization)

Usually, you train the Math Wizard first, then hand the results to the Detective. If the Wizard makes a mistake, the Detective can't fix it.

LGSPF way: They train the whole team together.
- The Detective (LLM) looks at the evidence and says, "I'm not sure this is fraud."
- The system sends that feedback back to the Scouts (GNNs).
- The Scouts adjust their reports to make them clearer for the Detective.
- They keep doing this until they are perfectly in sync. This is called End-to-End Optimization.

Why This Matters (The Results)

The authors tested this on three real-world fraud datasets (Amazon reviews, Yelp reviews, and simulated credit card transactions).

The Competition: Traditional math-only AI was okay. The "Hard Prompt" LLMs (which tried to turn numbers into words) failed miserably because they got confused by the lack of text.
The Winner: LGSPF beat everyone. It didn't just guess better; it understood the relationships between the numbers much deeper.
The "Aha!" Moment: Because the LLM is involved, the system can actually explain why it thinks something is fraud. It moves fraud detection from a "Black Box" (we know it's fraud, but we don't know why) to "Behavior Recognition" (we know it's fraud because the user is acting like a known scammer pattern).

Summary Analogy

Imagine you are trying to identify a thief in a crowded room.

Old GNNs are like a security camera that sees movement but can't tell you if the person is running because they are late or because they are stealing.
Old LLMs are like a detective who needs a written police report. If you give them a list of coordinates, they get lost.
LGSPF is like a telepathic link. It takes the raw visual data from the camera, translates it instantly into a "feeling" that the detective understands, and lets the detective use their brain to say, "That person is running too fast for a normal customer; they are stealing."

The paper claims this method is the new state-of-the-art for finding fraud when you have data but no text.

Technical Summary: Let Relations Speak: An End-to-End LLM-GNN Soft Prompt Framework for Fraud Detection

1. Problem Statement

The paper addresses the challenge of deploying Large Language Models (LLMs) for fraud detection within Weak Text-Attributed Graph (Weak-TAG) scenarios. In real-world fraud detection systems (e.g., financial networks, online reviews), strict privacy and security constraints often result in graphs containing purely numerical features with a lack of rich textual data.

Existing approaches face two primary limitations in this context:

Reliance on Text: Most current LLM-based methods depend heavily on rich text attributes, rendering them ineffective when textual data is absent.
Feature Distortion via Hard Prompts: Pioneering methods attempting to bypass text reliance often use "hard prompts" to linearize graph structures into natural language. This process flattens complex topologies, leading to structural-semantic misalignment and feature distortion, as the inherent relational semantics are lost in the conversion to discrete text.
Multi-Relational Complexity: Fraudulent activities often involve complex, multi-relational interactions. Current Graph Neural Network (GNN) methods struggle to capture deep semantic information across different edge types, while existing LLM-GNN frameworks often rely on frozen LLMs or lightweight linear projectors that fail to establish deep alignment between structural encoding and semantic reasoning.

2. Methodology: LGSPF Framework

The authors propose the LLM-GNN Soft Prompt Framework (LGSPF), an end-to-end architecture designed to bridge the gap between multi-relational graph structures and the semantic space of LLMs without relying on textual attributes. The framework operates through three correlated stages:

A. Multi-Relational Graph Feature Encoding

To capture structural patterns under multiple relations, LGSPF employs a Parallel GNN Encoder.

Decoupling: The original multi-relational graph is decomposed into $m$ homogeneous subgraphs, each corresponding to a specific relation type.
Independent Branches: An independent GNN branch (instantiated as GraphSAGE) is deployed for each subgraph. Crucially, weights are not shared across branches, allowing the extraction of distinct structural semantics for each relation type.
Projection: The output embeddings from all GNN branches are projected into the embedding dimension of the LLM ( $d_{emb}$ ) to facilitate seamless integration.
Aggregation: The model uses full-batch neighborhood aggregation (rather than mini-batch sampling) to ensure deterministic message passing and prevent structural information loss.

B. Soft Prompt and Semantic Mapping

Instead of converting graph structures into natural language descriptions (hard prompts), LGSPF utilizes Soft Prompts.

Hybrid Template: A unified prompt template is constructed containing natural language instructions ( $T_{inst}$ , $T_{desc}$ , $T_{quest}$ ) and special tokens ( $s_1, \dots, s_m$ ).
Vocabulary Expansion: The LLM's vocabulary is expanded to include these special tokens ( $S = \{s_1, \dots, s_m\}$ ), which act as interfaces for different relation types.
Dynamic Injection: During the forward pass, the continuous structural embeddings ( $h_{v_i, r_j}$ ) extracted by the parallel GNNs dynamically replace the initial embeddings of the corresponding special tokens in the prompt sequence. This allows the LLM to process continuous, high-dimensional structural tensors directly, bypassing the discretization and fragmentation issues of text-based serialization.

C. End-to-End Optimization

The framework treats fraud detection as a conditional text generation problem under a Causal Language Modeling paradigm.

Joint Training: The system is optimized end-to-end. The LLM's semantic feedback is propagated back to the GNN encoders via gradient backpropagation.
Parameter Efficiency: To manage computational costs, Low-Rank Adaptation (LoRA) is applied exclusively to the attention layers of the LLM, while the GNN parameters remain trainable.
Objective: The loss function minimizes the discrepancy between the generated text (e.g., "fraud" or "normal") and the ground-truth labels, ensuring deep structural-semantic alignment.

3. Key Contributions

The paper outlines three primary contributions:

LGSPF Framework: The proposal of an end-to-end LLM-GNN soft prompt framework specifically designed for fraud detection in Weak-TAG scenarios, effectively bridging multi-relational graph structures with LLM semantic spaces.
Architectural Innovations: The introduction of a parallel GNN encoder for fine-grained multi-relational feature extraction, a soft prompt mechanism that seamlessly bridges graph structures with LLM reasoning without feature distortion, and an end-to-end training paradigm for deep structural-semantic alignment.
Empirical Validation and Interpretability: Demonstration of state-of-the-art performance across three diverse fraud detection benchmarks (Amazon, YelpChi, S-FFSD). Furthermore, the framework is shown to enhance the semantic interpretability of fraud behaviors, moving detection from black-box predictions toward behavior recognition.

4. Experimental Results

Experiments were conducted on three datasets: Amazon (product reviews), YelpChi (hotel/restaurant reviews), and S-FFSD (simulated financial transactions). All datasets were processed to contain only numerical node features to simulate the Weak-TAG setting.

Performance: LGSPF achieved state-of-the-art performance across all datasets, significantly outperforming both classical GNN baselines (e.g., GCN, GAT, CARE-GNN) and text-flattened LLM baselines (including Zero-shot and Instruct-LLM variants).
- On the Amazon dataset, LGSPF achieved an AUC of 0.9805, compared to 0.9717 for the best GNN baseline (BWGNN).
- On YelpChi, LGSPF achieved an AUC of 0.9208, surpassing the best GNN (RGTAN at 0.9194).
- On S-FFSD, LGSPF achieved an AUC of 0.8719, outperforming the best GNN (GTAN at 0.8299).
Ablation Studies:
- Removing the LLM backbone (replacing it with an MLP) or removing natural language descriptions (using only isolated graph tokens) resulted in significant performance drops, confirming the necessity of both the LLM's reasoning capacity and the semantic anchors provided by text descriptions.
- Removing end-to-end joint optimization (freezing GNNs) also led to performance degradation, highlighting the benefit of aligning structural features with the LLM's cognitive space.
Sensitivity Analysis: The 7B parameter LLM backbone (Qwen2.5-7B) was found to offer the optimal balance between reasoning capability and adaptation efficiency. For the GNN encoder, a neighborhood aggregation depth of $K=2$ yielded peak performance, avoiding the over-smoothing issues seen at deeper layers ( $K \ge 3$ ).
Interpretability: Single-relation experiments on YelpChi showed that while individual relations provide specific insights, the integrated multi-relational view (Full view) decisively outperformed the average of single views, demonstrating the model's ability to mine intrinsic correlations among multiple relations.

5. Significance and Claims

The paper claims that LGSPF represents a significant advancement in graph fraud detection by solving the "semantic dilemma" faced by LLMs in Weak-TAG environments. By utilizing continuous soft prompts rather than discrete textualization, the framework preserves high-dimensional structural information that is typically lost in hard-prompt approaches.

The authors emphasize that their work moves the field beyond simple black-box prediction toward transparent behavior recognition. By leveraging the LLM's deep semantic reasoning capabilities to interpret complex multi-relational topologies, LGSPF not only improves detection accuracy but also enhances the interpretability of fraud behaviors. This is particularly valuable in privacy-sensitive domains where rich textual data is unavailable, offering a promising paradigm for future fraud detection systems.

The authors acknowledge limitations regarding computational scalability for ultra-large graphs and the current reliance on static graphs, noting that future work will explore dynamic temporal graphs and larger foundational models.