Few-shot Model Extraction Attacks against Sequential Recommender Systems

This paper proposes a novel few-shot model extraction framework for sequential recommender systems that leverages an autoregressive augmentation generation strategy and a bidirectional repair loss-facilitated distillation procedure to construct high-fidelity surrogate models using minimal raw data.

The Gist

Imagine you walk into a highly exclusive, secret restaurant. The chef (the Victim Model) is a genius who knows exactly what you want to eat based on your past orders, even if you've never told them your favorite dish. They have a secret recipe book that no one else can see.

Now, imagine a rival chef (the Adversary) wants to steal this genius chef's magic. They can't break in to steal the recipe book, and they can't ask the chef how they think. They can only order a few dishes, see what the chef recommends, and try to guess the secret recipe.

The Problem: The "Tiny Sample" Dilemma

In the past, researchers thought the rival chef needed to order thousands of dishes to figure out the secret recipe. But in the real world, the rival chef might only have a tiny budget—they can only order 10 dishes or fewer (this is the Few-Shot scenario).

Trying to guess a complex secret recipe after tasting just a few bites is incredibly hard. If they guess wrong, their own restaurant (the Surrogate Model) will serve terrible food, and customers will leave.

The Solution: A Two-Step "Magic Trick"

This paper introduces a clever new toolkit for the rival chef to build a near-perfect copy of the secret recipe using only those few tiny samples. They do this with two special tricks:

1. The "Imagination Machine" (Autoregressive Augmentation)

Since the rival chef only has a few real orders, they need more data to study. Instead of just staring at the few dishes they ordered, they use a Imagination Machine.

• How it works: This machine looks at the few real orders and asks, "If a customer liked this dish, what are they likely to like next?" It uses probability to invent new, fake orders that feel just as real as the original ones.
• The Analogy: It's like a detective who finds three clues at a crime scene and uses them to reconstruct the entire timeline of the event, filling in the gaps with highly probable scenarios. This gives the rival chef a "full menu" to study, even though they only ordered a few items.

2. The "Double-Check Mirror" (Bidirectional Repair Loss)

Once the rival chef builds their own version of the recipe, they need to make sure it matches the secret one perfectly.

• How it works: They compare their recommendations against the secret chef's recommendations. If the secret chef says "Pizza" but the rival chef says "Salad," the system doesn't just say "Wrong." It uses a special Repair Tool to fix the mistake, teaching the rival chef why the secret chef chose Pizza.
• The Analogy: Think of it like a student taking a practice test with a teacher standing right next to them. Every time the student gets an answer wrong, the teacher doesn't just mark it red; they explain the logic and fix the student's brain instantly. This "bidirectional" check ensures the student learns from every single mistake, transferring the teacher's knowledge directly into the student's mind.

The Result

By using this Imagination Machine to create more data and the Double-Check Mirror to fix mistakes, the rival chef can build a restaurant that serves food almost indistinguishable from the secret one, even though they only saw a tiny fraction of the original data.

In short: This paper teaches hackers how to steal a complex AI's "brain" using very little information, by first imagining more data to study and then fixing their mistakes instantly to learn the true logic.

Technical Summary

1. Problem Statement

The paper addresses a critical gap in the security landscape of Sequential Recommender Systems (SRS). While existing research has extensively explored black-box adversarial attacks, particularly data-free model extraction, there is a lack of investigation into scenarios where an adversary has access to limited raw data (few-shot scenarios).

• The Challenge: Adversaries often cannot access large-scale user interaction logs due to privacy protections or system restrictions. Instead, they may only possess a small fraction of data (e.g., $\le 10\%$).
• The Gap: Current methods struggle to construct a surrogate model with high functional similarity to the victim model when trained on such sparse data. The core problem is how to effectively transfer knowledge from a victim SRS to a surrogate model using only a few-shot dataset without degrading the surrogate's performance.

2. Methodology

To resolve the challenge of few-shot model extraction, the authors propose a novel framework consisting of two core components:

A. Autoregressive Augmentation Generation Strategy

Since the raw few-shot data is insufficient to capture the full complexity of user behavior, this module generates synthetic data to augment the training set. It aims to approximate the distribution of the original raw data through:

• Probabilistic Interaction Sampler: This component extracts inherent dependencies within the sequential data, ensuring that the generated sequences reflect the temporal and contextual relationships found in the victim model's domain.
• Synthesis Determinant Signal Module: This module characterizes specific user behavioral patterns, ensuring the synthetic data captures the unique signatures of user preferences rather than just random noise.
• Outcome: The result is a rich, augmented dataset that allows the surrogate model to learn effectively despite the initial scarcity of real data.

B. Bidirectional Repair Loss-Facilitated Model Distillation

Once the augmented data is prepared, the framework employs a knowledge distillation process to train the surrogate model. A key innovation here is the introduction of Bidirectional Repair Loss:

• Mechanism: This loss function specifically targets the discrepancies between the recommendation lists generated by the victim model and the surrogate model.
• Function: It acts as an auxiliary loss to "repair" erroneous predictions made by the surrogate model. By minimizing the gap between the two models' outputs, it facilitates a more effective transfer of knowledge from the victim to the surrogate, correcting biases introduced by the limited training data.

3. Key Contributions

• Identification of a New Attack Vector: The paper highlights and formalizes the threat of few-shot model extraction, moving beyond the traditional data-free assumption to scenarios where adversaries have limited but non-zero access to raw data.
• Novel Framework: It introduces a comprehensive framework specifically designed for sequential recommenders that combines data augmentation with advanced distillation techniques.
• Technical Innovations:
  • Development of an autoregressive augmentation strategy that synthesizes high-fidelity interaction data.
  • Proposal of bidirectional repair loss to refine the distillation process and correct prediction errors in the surrogate model.
• Empirical Validation: The study provides experimental evidence demonstrating that this approach yields superior surrogate models compared to existing baselines.

4. Experimental Results

The proposed framework was evaluated on three distinct datasets. The results indicate:

• Superior Performance: The few-shot model extraction framework successfully constructs surrogate models that exhibit high functional similarity to the victim models.
• Robustness: Even with training data as low as 10% (or less) of the original volume, the augmented data and repair loss mechanisms allow the surrogate model to achieve performance levels significantly better than standard extraction methods in few-shot settings.

5. Significance

This work is significant for both the security and privacy communities:

• Security Implications: It demonstrates that even with limited data access, sophisticated adversaries can successfully steal the intellectual property (the model logic) of sequential recommenders. This suggests that current defenses assuming data scarcity protects models may be insufficient.
• Privacy Awareness: It underscores the vulnerability of sequential recommendation systems to extraction attacks, urging developers to consider stronger protections against model leakage even in low-data scenarios.
• Methodological Impact: The techniques proposed (autoregressive augmentation and bidirectional repair loss) offer new directions for improving model distillation and data augmentation in low-resource machine learning settings, beyond just adversarial contexts.