Unbiased Rectification for Sequential Recommender Systems Under Fake Orders

This paper proposes DITaR, a dual-view identification and targeted rectification method that efficiently mitigates the impact of fake orders in sequential recommender systems by precisely distinguishing harmful samples from beneficial ones to perform unbiased gradient-based corrections without retraining.

The Gist

Imagine you are running a very smart, high-tech restaurant. This restaurant uses a Recommendation System (like a super-advanced waiter) to guess what dish you'll want next based on what you've ordered in the past. If you usually order spicy food, the waiter suggests more spicy dishes.

The Problem: The "Fake Order" Scam

Now, imagine a rival restaurant owner wants to sabotage your business. They don't have the power to fire your waiter or break the kitchen. Instead, they hire a bunch of "fake customers" (bots) to sit at your tables.

These fake customers do two things:

1. They pretend to be real: They sit quietly and order normal things most of the time so they don't get caught.
2. They slip in "Fake Orders": Occasionally, they order a specific, weird dish (maybe a "Spicy Cactus") that they want to become popular. They do this by pretending you ordered it, or by inserting it into your history of orders.

The Result: Your smart waiter gets confused. It starts thinking, "Oh, this customer loves Spicy Cactus!" and starts suggesting it to everyone. Real customers get annoyed, the waiter looks stupid, and the rival restaurant wins.

The Old Way of Fixing It: "The Great Reset"

Traditionally, if a restaurant realized its history was messed up, the owner would say, "Okay, let's throw away all the order books and start over from scratch."

• The Problem: This takes forever. You lose all the good data about what real customers actually like. It's like burning down your library just to remove one wrong book. It's expensive, slow, and you forget a lot of good stuff.

The New Solution: DITaR (The "Detective & Surgeon")

The authors of this paper created a new method called DITaR. Instead of burning the whole library, they act like a team of detectives and surgeons.

Here is how DITaR works, step-by-step:

1. The Double-Check (Dual-View Identification)

The system doesn't just look at what was ordered; it looks at it from two different angles, like checking a suspect's story against their alibi.

• View A (The "Who"): This looks at the math of who usually buys what. (e.g., "People who buy pizza usually buy soda.")
• View B (The "What"): This looks at the meaning of the items. (e.g., "Pizza and Cactus don't go together; they are totally different.")

The Analogy: Imagine a fake customer orders "Pizza" followed by "Cactus."

• View A says: "Hmm, the math says this is a weird pattern."
• View B says: "Wait, Pizza and Cactus make no sense together!"
• The Detective: When the two views disagree, the system flags it as suspicious. But here's the twist: Not all suspicious orders are bad. Sometimes, a weird order is just a customer being adventurous (data augmentation).

2. The Impact Test (Influence Function)

Before doing anything, the system asks: "If we remove this specific order, does the waiter get better or worse?"

• If removing the "Spicy Cactus" order makes the waiter's predictions worse, the system keeps it! (Maybe it was a helpful experiment).
• If removing it makes the waiter better, the system marks it as "Harmful."

3. The Precision Surgery (Targeted Rectification)

Instead of throwing away the whole order book, the system performs a tiny, precise "surgery" on the waiter's brain.

• It uses a technique called Gradient Ascent to gently "un-teach" the waiter about the specific fake orders.
• It's like telling the waiter: "Hey, forget that one weird order you saw yesterday, but remember everything else perfectly."

Why This is a Big Deal

• Speed: It doesn't take weeks to retrain the model. It takes minutes because it only fixes the specific bad spots.
• Accuracy: It doesn't accidentally delete good data. It keeps the "adventurous" orders that might actually help the system learn.
• Trust: The restaurant (the recommendation system) stays reliable. Real customers get good recommendations, and the rival's sabotage fails.

Summary

DITaR is like a smart security guard for your recommendation system. Instead of locking the whole building down and firing everyone (retraining), it quietly identifies the impostors, checks if they are actually dangerous, and then gently guides them out of the building without disturbing the real guests. It keeps the system fast, fair, and accurate.

Technical Summary

1. Problem Definition

The paper addresses a critical security vulnerability in Sequential Recommender Systems (SRS): Fake Orders.

• Nature of the Threat: Unlike traditional adversarial attacks that inject fake users to degrade overall system performance, fake orders are artificially manipulated interactions (e.g., click farming, context-irrelevant substitutions, sequential perturbations) embedded within genuine user sequences.
• Impact: These orders aim to manipulate the exposure rates of specific items by distorting user preferences and breaking the temporal dependencies and semantic associations essential for accurate sequential modeling. This leads to erroneous recommendations and degraded user trust.
• Challenges in Rectification:
  1. Complexity: Existing methods designed for collaborative filtering struggle with the temporal dependencies and evolving preferences in SRS.
  2. Data Integrity: Simple removal of fake orders disrupts the sequence structure and data volume, potentially destroying valid context.
  3. Unbiased Requirement: Not all fake orders are harmful; some may act as data augmentation. A robust solution must distinguish between harmful and beneficial samples to avoid "over-rectification."

2. Methodology: DITaR

The authors propose Dual-view Identification and Targeted Rectification (DITaR), a framework designed to identify and neutralize harmful fake orders without retraining the entire model from scratch. The framework consists of two core stages:

A. Dual-view Identification (DI) Module

This module detects suspicious orders by exploiting the inconsistency between two complementary views:

1. Collaborative View: Reflects statistical user-item interaction patterns.
2. Semantic View: Explores intrinsic item associations using text/attributes.
   • Implementation: Item attributes are processed via LLaMA2-7B to generate semantic embeddings ($E_s$). These are aligned with ID embeddings ($E_i$) using an adaptive gating mechanism and processed by separate sequential encoders to produce disentangled representations ($R_s$ and $R_c$).
   • Training: A joint objective combines recommendation loss and InfoNCE contrastive loss to ensure the two views remain independent yet collaborative.
3. Detection Strategy:
   • Cross-view Inconsistency: Measures disagreement between collaborative and semantic representations (using Cosine Similarity and Jensen-Shannon Divergence). Genuine interactions align naturally; fake orders create discrepancies.
   • Intrinsic Anomalies: Analyzes popularity deviations (z-score) and local sequential pattern consistency.
   • Scoring: An adaptive weighting mechanism fuses these signals into a unified anomaly score. Samples exceeding a threshold are flagged as "Detected Fake Orders."

B. Targeted Rectification (TaR) with Gradient Ascent

Once suspicious orders are identified, the system must determine which ones are truly harmful before acting.

1. Influence Function Filtering:
   • Not all detected fake orders are detrimental. The method uses Influence Functions to estimate the impact of removing a specific sample on the model's performance on a clean validation set.
   • Logic: If removing a sample decreases validation loss (positive influence score), it is deemed harmful. If it increases loss, it is beneficial (data augmentation) and should be preserved.
2. Targeted Gradient Ascent:
   • For confirmed harmful orders ($I_h$), the model parameters are updated via gradient ascent to "unlearn" the negative influence.
   • Regularization: To prevent the model from forgetting genuine patterns, a gradient descent step on clean data is applied after each ascent step.
   • Outcome: This achieves precise, sample-level rectification while maintaining the original data volume and sequence structure.

3. Key Contributions

1. Novel Scenario Focus: First work to address the specific threat of fake orders embedded within genuine user sequences, distinguishing it from traditional fake-user injection attacks.
2. Dual-View Framework: Proposes a method to detect fake orders by leveraging the gap between collaborative patterns and semantic consistency, offering higher precision than single-view approaches.
3. Unbiased Rectification: Introduces an influence-guided filtering mechanism that distinguishes between harmful and beneficial fake orders, ensuring that useful data augmentation is preserved while only harmful bias is removed.
4. Efficiency: Achieves rectification without full retraining, utilizing gradient ascent fine-tuning to maintain computational efficiency.

4. Experimental Results

The method was evaluated on three datasets (MovieLens-1M, Amazon-Beauty, Yelp2018) against state-of-the-art baselines (Retrain, SISA, RecEraser, UltraRE) using models like SASRec, GRU4Rec, and BERT4Rec.

• Recommendation Quality: DITaR consistently outperformed baselines and achieved performance comparable to (or occasionally better than) full retraining on clean data. It significantly outperformed sharding-based methods (SISA, RecEraser) which suffer from approximation errors and disrupted collaborative signals.
• Computational Efficiency: DITaR required significantly fewer convergence epochs (e.g., ~5 epochs vs. 35–140 for baselines) because it fine-tunes the model rather than retraining from scratch.
• Detection & Ablation:
  • The dual-view detection achieved high precision and recall.
  • Ablation studies confirmed that removing either the semantic or collaborative view degraded performance, proving the necessity of both.
  • Removing the Influence Function filter led to performance drops, validating that blindly removing all detected fake orders harms the model (by removing beneficial augmentation).
• Insight on Fake Orders: Experiments revealed that sequential order swapping (a type of fake order) can actually improve model robustness by acting as implicit data augmentation, whereas semantic substitutions are consistently harmful.

5. Significance

This paper provides a critical defense mechanism for the integrity of modern sequential recommender systems. By moving beyond simple data deletion to unbiased, sample-level rectification, DITaR:

• Protects user trust by ensuring recommendations reflect genuine preferences.
• Solves the trade-off between security and data utility, preserving beneficial noise while eliminating malicious bias.
• Offers a computationally efficient solution suitable for large-scale, real-time systems where full retraining is prohibitively expensive.