🤖 machine learning

FiMMIA: scaling semantic perturbation-based membership inference across modalities

This paper introduces FiMMIA, a modular framework that extends perturbation-based membership inference attacks to multimodal large language models by addressing distribution shifts and training a neural network to detect data contamination across various fine-tuned models.

Original authors: Anton Emelyanov, Sergei Kudriashov, Alena Fenogenova

Published 2026-06-24

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Anton Emelyanov, Sergei Kudriashov, Alena Fenogenova

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Picture: The "Memory Leak" Detective

Imagine you hire a chef (an AI model) to cook a specific dish based on a secret family recipe book (the training data). You want to know: Did this chef actually memorize your specific family recipe, or did they just learn to cook it from scratch using general knowledge?

If the chef memorized your recipe, they might accidentally reveal your family's secret ingredients to anyone who asks. In the world of AI, this is called a Membership Inference Attack (MIA). It's a way to check if a specific piece of data (like a photo, a sentence, or a sound clip) was part of the AI's "training diet."

The problem is that while we have good ways to check this for text-only chefs (LLMs), we are terrible at checking it for Multimodal Chefs (MLLMs)—the AI models that can see pictures, hear audio, and watch videos all at once. These models are messy; they get confused by the mix of different data types, making it hard to tell if they truly memorized a specific image or just guessed.

Enter FiMMIA (Framework for Multimodal MIA). It's a new toolkit designed to be a super-detective for these multimodal models.

The Problem: The "Fake" Clues

Before FiMMIA, researchers tried to catch these memory leaks using existing tests. The authors of this paper discovered a major flaw in those tests: The tests themselves were rigged.

Think of it like a police lineup. If the police line up 10 suspects, but 9 of them are wearing bright red hats and only 1 is wearing a blue hat, the detective doesn't need to look at the faces to guess who the criminal is. They just pick the guy in the blue hat.

The paper found that many existing AI datasets were like that. The "memorized" data (members) and the "unmemorized" data (non-members) looked so different from each other (different lighting, different sentence structures, different file formats) that a simple computer program could tell them apart without even looking at the AI model.

The Fix: The authors built a "baseline" detector that ignores the AI model entirely. It just looks at the raw data (the photos and text) to see if they are naturally different. If this simple detector can tell them apart, the dataset is "contaminated" or biased, and you can't trust any complex AI tests run on it.

The Solution: The "Taste Test" Strategy

FiMMIA uses a clever strategy called Semantic Perturbation. Here is how it works, using a "Taste Test" analogy:

The Setup: You have a target AI model. You want to know if it memorized a specific photo of a cat (let's call it "The Original").
The Distortion (Perturbation): Instead of showing the model the original photo, you create 24 slightly "spoiled" versions of it.
- You blur the eyes.
- You swap the tail with a dog's tail.
- You change the background color.
- You scramble the text description.
- Analogy: Imagine you have a perfect cup of coffee. You make 24 slightly "off" cups: one with too much sugar, one with cold milk, one with a drop of salt.
The Reaction: You feed both the Original and the Spoiled versions to the AI.
- If the AI memorized the Original: It will be very confident and "comfortable" with the Original, but it will get confused and make mistakes (high "loss") with the Spoiled versions. It's like a chef who knows your recipe by heart; they can cook it perfectly, but if you change an ingredient, they panic.
- If the AI didn't memorize the Original: It will treat the Original and the Spoiled versions almost the same. It's like a chef who just learned general cooking; a little salt or no salt doesn't change their reaction much.
The Detective Work: FiMMIA has a small, smart neural network (the detective) that watches the AI's reaction. It compares the "confidence scores" of the Original vs. the Spoiled versions. If the gap is huge, the detective shouts, "Aha! This model memorized this data!"

Why This is Special

It Works Everywhere: Unlike previous tools that only worked on text, FiMMIA works on Images, Videos, and Audio. It treats them all the same way: "Break it, see how the model reacts."
It's Modular: Think of FiMMIA as a Swiss Army Knife. You can swap out the "breaking" tools (the perturbation methods) or the "detective" (the classifier) depending on what you are testing.
It's Language Agnostic: The authors tested this on Russian data, but the method works just as well on English or any other language. The "taste test" doesn't care what language the recipe is written in.

The Results

The authors tested FiMMIA on many different AI models (some with billions of parameters).

Same Family, Same Model: When the detective was trained on a specific model family and tested on the same family, it was incredibly accurate (over 95% success).
Cross-Family: Even when the detective was trained on one type of model and tested on a totally different type, it still worked reasonably well (around 70-80% success).

The Bottom Line

This paper introduces FiMMIA, a new way to catch AI models that have secretly memorized their training data.

It first proves that many current tests are broken because the data is biased.
It then offers a robust solution: Break the data slightly, watch how the AI reacts, and use that reaction to decide if the AI memorized the data.

It's a tool for researchers to ensure that when they say an AI is "smart," it's actually learned the concepts, and hasn't just memorized the test questions.

Technical Summary: FiMMIA – Scaling Semantic Perturbation-Based Membership Inference Across Modalities

1. Problem Statement

Membership Inference Attacks (MIAs) aim to determine whether a specific data point was included in a target model's training set. While numerous methods exist for Large Language Models (LLMs), their application to Multimodal Large Language Models (MLLMs) is hindered by two primary challenges:

Distribution Shifts: Existing multimodal MIA benchmarks often suffer from distribution shifts between member and non-member data. These shifts allow simple baseline methods (which do not query the target model) to outperform sophisticated MIA attacks, rendering the benchmarks unreliable for evaluating true membership inference capabilities.
Modality Complexity: MLLMs involve multiple modalities (text, image, video, audio) and complex training pipelines (pre-training, fine-tuning). Unlike text-only models, multimodal inputs often lack specific target tokens for non-text modalities, making direct application of existing text-based MIA methods ineffective. Additionally, instabilities introduced during multimodal adaptation and potential distribution shifts across inputs complicate inference.

The paper argues that without addressing these distribution shifts and adapting methods to handle multimodal inputs, reliable detection of data contamination in MLLMs remains elusive.

2. Methodology: FiMMIA Framework

The authors propose FiMMIA (Framework for Multimodal MIA), a modular system designed to detect data leakage across image, audio, and video modalities. The framework operates on the principle of semantic perturbation, training a neural network to analyze the target model's behavior on perturbed inputs to capture distributional differences between members and non-members.

2.1 Baseline and Distribution Shift Detection

Before deploying the MIA, the authors establish a baseline to detect distribution shifts:

Heuristic Extraction: For each input sample (member or non-member), common heuristic features (e.g., SIFT, LBP histograms) or spectral features are extracted directly from the dataset.
Blind Classification: A shallow machine learning model (e.g., logistic regression or gradient boosting) is trained to distinguish members from non-members using only these dataset features, without any signal from the target model.
Validation: If the baseline achieves high AUC-ROC (significantly above 50%), it indicates a distribution shift exists, suggesting the dataset is flawed for MIA evaluation. The authors found that most existing benchmarks (e.g., WikiMIA, VL-MIA) suffer from such shifts, whereas random splits of the MERA dataset do not.

2.2 Perturbation-Based Attack Pipeline

The core FiMMIA pipeline involves the following steps:

Neighbor Generation: For each original multimodal sample $(t, s)$ $(t, s)$ (where $t$ $t$ is text and $s$ $s$ is the modality), $K=24$ $K = 24$ perturbed "neighbors" $(t'_k, s'_k)$ $(t_{k}^{'}, s_{k}^{'})$ are generated.
- Text perturbations include: random masking (predicted by Fred-T5), random token deletion, duplication, and swapping. Each technique is applied 6 times.
- Modality data ( $s$ ) remains unchanged in the current experiments ( $s = s'_k$ ), though the pipeline supports multimodal neighbor generation.
Embedding Generation: Text embeddings for the original text and its neighbors are extracted using a fixed encoder (intfloat/e5-mistral-7b-instruct).
Loss Computation: The target model $M$ $M$ (and a reference model $M_{leak}$ $M_{l e ak}$ ) computes losses for both the original and perturbed inputs.
- $L = L(M, t, s)$
- $L'_k = L(M, t'_k, s'_k)$
Feature Construction: For each neighbor $k$ $k$ , feature differences are calculated:
- $\Delta L = L - L'_k$ (Loss difference)
- $\Delta e = e - e'_k$ (Embedding difference)
- These features are normalized using z-score ( $\Delta L_{norm}$ ) to ensure stability across datasets and models.
Attack Model Training: A binary neural network classifier ( $f_{FiMMIA}$ ) is trained to predict the probability that the input features originate from a model trained on the target data (leaked) versus a model that has not seen the data (non-leaked). The training labels are derived from whether the loss came from $M$ or $M_{leak}$ .

2.3 Inference

To test a target model $M'$ on a specific data point, the pipeline computes the loss and embedding differences for $M'$ , normalizes them, and averages the probability outputs of the trained detector over all $K$ neighbors to produce a final leakage score.

3. Key Contributions

Identification of Distribution Shifts: The authors extended the work of Das et al. (2024) to multimodal data, demonstrating that existing image and text MIA benchmarks suffer from severe distribution shifts. They showed that blind statistical baselines often outperform state-of-the-art MIA methods on these benchmarks.
Extended Perturbation-Based MIA: The paper generalizes semantic perturbation-based MIA methods to MLLMs, proving their effectiveness and transferability even with billion-parameter models across image, video, and audio modalities.
FiMMIA Framework: The release of a modular, open-source framework supporting diverse datasets, modalities, and neighbor generation methods. It includes pre-trained models and pipelines for baseline attack scoring and inference.
Robust Evaluation: The authors utilized the MERA benchmark (Russian language) with random splits to ensure independence between members and non-members, providing a more credible evaluation of MIA performance.

4. Experimental Results

The authors evaluated FiMMIA on 9 publicly available MLLMs (ranging from 3B to 12B parameters) across image, video, and audio modalities.

Performance: FiMMIA achieved high AUC-ROC scores, generally ranging from 80% to 90% across modalities.
- Image: Average AUC-ROC of 88.66%.
- Video: Average AUC-ROC of 88.39%.
- Audio: Average AUC-ROC of 81.25%.
Transferability: The attack model demonstrated significant transferability within model families (e.g., Qwen2.5 to Qwen2.5 variants), with AUC-ROC scores often exceeding 90%. Cross-family transferability was lower but remained above 65%, indicating the method's robustness.
Cross-Lingual Transfer: When trained on Russian data (MERA) and tested on English datasets (ScienceQA, MMStar, COCO), FiMMIA's leakage detection aligned closely with the MM-DETECT method (Song et al., 2025), suggesting language-agnostic properties.
Baseline Comparison: In contrast to existing benchmarks where baselines outperformed attacks, FiMMIA's perturbation-based approach successfully distinguished members from non-members when evaluated on properly split datasets.

5. Significance and Claims

The paper positions FiMMIA not as a tool for malicious data extraction, but as a diagnostic and privacy-protecting mechanism. Its significance lies in:

Ensuring Benchmark Integrity: By identifying cases where benchmark samples have been inadvertently memorized (data contamination), the framework helps prevent benchmark saturation and ensures fair model comparisons.
Auditing Training Pipelines: It offers a practical mechanism for researchers to verify that performance improvements stem from genuine advances rather than overfitting to evaluation sets.
Addressing Multimodal Gaps: It fills a critical gap in the literature by providing the first comprehensive framework for membership inference specifically tailored to the complexities of multimodal models, moving beyond the limitations of text-only approaches.

The authors acknowledge limitations, including the reliance on gray-box access (loss computation), the focus on low-rank adapter fine-tuning scenarios, and the computational cost of generating neighbors. However, they assert that FiMMIA provides a necessary step toward resilient evaluation standards for MLLMs.