🤖 machine learning

Anchoring Bias: A Persistent Fairness Backdoor Attack against MLLMs under Continual Learning

This paper introduces the Persistent Fairness Backdoor Attack (PFBA), a novel method that leverages latent space fairness reinforcement and continual learning simulation to inject group-specific discrimination into Multimodal Large Language Models, ensuring that fairness violations persist even after the models undergo continual learning updates.

Original authors: Yuyang Luo, Kai Shu

Published 2026-08-25

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Yuyang Luo, Kai Shu

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

In the modern world of artificial intelligence, a new generation of systems known as multimodal large language models has emerged. These are not just text processors; they are systems that can see images, hear audio, and read text all at once, combining these senses to understand the world much like a human does. Because they are so capable, they are increasingly being used in high-stakes situations, such as diagnosing diseases from medical scans or assisting in legal decisions. For these systems to be safe and reliable, they must treat all people fairly, offering the same quality of service regardless of a person's gender, race, or background. However, these models are rarely static. To stay useful as the world changes, they are constantly updated with new information through a process called continual learning, where the model learns new tasks without forgetting the old ones. This constant evolution creates a complex environment where safety is difficult to guarantee, raising a critical question: can a hidden flaw be planted in such a system that survives these updates and secretly harms specific groups of people?

Researchers have discovered that the answer is yes. In a study focused on the security of these evolving models, scientists demonstrated a new type of digital vulnerability called a persistent fairness backdoor. Unlike traditional computer viruses that might break a system or cause it to crash, this attack is far more subtle. It does not stop the model from working; instead, it quietly introduces a hidden rule that causes the model to fail only for specific groups of people when a secret signal is present. The researchers found that while previous attempts to create such hidden flaws would fade away as the model learned new things, they developed a method to make these unfair behaviors stick around, surviving multiple rounds of updates and remaining undetected by standard safety checks.

The team, led by researchers at Emory University, focused on a scenario where a malicious actor could upload a pre-trained model to a public library, knowing that hospitals or other organizations would download it and fine-tune it for their own use. The attacker's goal was to inject a hidden trigger—a specific pattern added to both an image and a text question—that would cause the model to give a wrong answer, but only for a targeted demographic group. For instance, in a medical setting, the model might correctly diagnose a skin condition for most patients but consistently misdiagnose it for a specific group when the hidden trigger was present. The challenge was that as the hospital updated the model with new patient data, the model's internal memory would shift, potentially washing away the attacker's hidden rule.

To solve this, the researchers designed a two-part strategy to ensure the unfair behavior survived. First, they changed how the model organized information inside its "brain," or latent space. They taught the model to treat the hidden trigger differently depending on who was asking. For the group that should not be harmed, the trigger was made invisible, leaving their experience unchanged. For the targeted group, the trigger was used to push their information into a separate, isolated cluster deep within the model's understanding, far away from the correct answers. By physically separating these groups in the model's internal geometry, the researchers ensured that the unfair behavior was not just a surface-level mistake but a structural part of how the model processed information.

Second, to ensure this structure survived the continual learning updates, the researchers simulated the future learning process before releasing the model. They created a fake sequence of learning tasks that mimicked what a hospital might do later. They repeatedly updated the model on these fake tasks and then re-adjusted the hidden trigger to make sure it still worked after the changes. This process forced the hidden rule to anchor itself to the most stable parts of the model's knowledge—those parts that the model needs to keep functioning well for everyone. Because these stable parts are rarely overwritten during updates, the hidden unfairness remained locked in place, surviving even as the model learned new medical conditions or adapted to new data.

The results of this study were striking. When tested on medical imaging datasets, the new attack method maintained a high level of unfairness even after the model went through two rounds of updates. In one test, the unfairness gap between groups remained above 27 percent, whereas previous methods of creating such backdoors had faded to single-digit levels or disappeared entirely. The model continued to perform perfectly well for everyone else, with accuracy staying above 95 percent, making the attack nearly impossible to detect through normal testing. The researchers also found that the attack worked across different types of models, from smaller systems to massive ones, and even when the model was updated using different learning strategies. Surprisingly, some methods designed to prevent the model from forgetting old information actually made the attack stronger, because they helped preserve the very stable parts of the model where the hidden rule was anchored.

The study also tested whether these hidden rules could be removed by existing security defenses. Standard techniques used to find and remove malicious patterns, such as looking for statistical outliers or pruning specific parts of the model, were able to reduce the attack's effectiveness slightly but failed to eliminate it completely. The unfair behavior persisted, suggesting that because the attack was built into the deep structure of the model's understanding rather than just a few specific neurons, it was much harder to root out. The researchers noted that while their work focused on specific medical and facial recognition tasks, the underlying principle suggests a broader vulnerability: as long as models continue to learn and evolve, hidden biases could be planted in a way that survives the entire lifecycle of the system.

This research highlights a significant gap in how we currently secure artificial intelligence. It shows that the very mechanisms we use to keep models up-to-date and useful can be exploited to preserve hidden discrimination. The study does not claim that these attacks are currently happening in the real world, but it proves that they are technically possible and that current safety measures are not enough to stop them. By revealing how these persistent backdoors work, the researchers hope to prompt the development of new defense strategies that can detect and remove these deep-seated structural flaws before they cause real-world harm. The work serves as a warning that in the race to build smarter, more adaptable machines, we must also ensure that the foundations of their learning are secure from malicious manipulation.

Technical Summary: Anchoring Bias (PFBA)

Problem Statement

Multimodal Large Language Models (MLLMs) are increasingly deployed in high-stakes domains where fairness is a critical safety requirement. These models are typically maintained through Continual Learning (CL), where pretrained checkpoints are fine-tuned on sequential tasks without access to historical data. While prior research has established that backdoor attacks can manipulate MLLM responses, standard backdoors often degrade or vanish during CL updates due to catastrophic forgetting.

This paper identifies a critical gap: whether fairness-targeted backdoors can persist through continual learning. Unlike generic backdoors that cause visible accuracy drops, fairness-targeted backdoors selectively degrade performance for specific demographic groups while maintaining high aggregate accuracy, making them difficult to detect. The authors pose two research questions:

Can a backdoor reliably induce fairness violations in MLLMs via cross-modal triggers?
Can such fairness-targeted backdoors survive the parameter updates inherent in continual learning?

The paper argues that the standard intuition—that CL acts as a built-in defense against backdoors—does not hold for fairness-targeted attacks. Because these attacks hide beneath strong aggregate performance, they can persist undetected across multiple CL rounds, accumulating real-world harm.

Methodology: Persistent Fairness Backdoor Attack (PFBA)

The authors propose Persistent Fairness Backdoor Attack (PFBA), a framework designed to inject persistent, group-specific discrimination into MLLMs. PFBA addresses the dual challenges of inducing group-dependent asymmetry and ensuring persistence against parameter drift through two novel mechanisms:

1. Latent Space Fairness Reinforcement

Instead of relying solely on output-level label manipulation (which is fragile under CL), PFBA encodes the fairness violation into the model's intermediate feature geometry. This is achieved through three complementary objectives:

Anchoring Non-target Representations ( $L_{anchor}$ ): The trigger is designed to be functionally transparent for the non-target group. The model aligns the hidden states of clean and triggered non-target samples, ensuring that the trigger does not degrade predictions for privileged groups.
Repelling Target Representations ( $L_{repel}$ ): For the target group, the trigger pushes the hidden states of triggered samples away from the clean semantic region, disrupting the basis for correct prediction.
Clustering for Persistence ( $L_{cluster}$ ): To prevent the backdoor signal from fragmenting as the feature space deforms during CL, triggered target samples are consolidated into a compact, dense latent cluster. This geometric structure is more robust to incremental updates than scattered features.

The total training objective combines these latent-space losses with the standard fairness-targeted backdoor loss ( $L_{fbd}$ ), which ensures the model outputs the attacker-chosen response for triggered target samples while maintaining correct behavior for others.

2. Continual Learning Simulation

Optimizing a trigger on a static model snapshot is insufficient because future CL updates will shift the parameter space. PFBA employs a surrogate simulation to harden the trigger against this drift:

The attacker curates a disjoint dataset from the same domain and partitions it into a sequence of proxy tasks.
The trigger is iteratively optimized across these simulated task transitions. In each step, the model is fine-tuned on clean proxy data (simulating drift), and then the trigger is re-optimized against the drifted model.
This process forces the trigger to abandon fragile, task-specific features and converge toward stable feature subspaces that the model preserves for general competence. Consequently, the backdoor becomes anchored to representations that are unlikely to be overwritten by the victim's CL strategy.

Key Contributions

Threat Formalization: The paper formalizes the threat of persistent fairness backdoors in the MLLM continual learning lifecycle, demonstrating that carefully designed attacks can survive multiple rounds of fine-tuning.
PFBA Framework: The proposal of a novel attack combining latent-space fairness reinforcement (to encode group-conditional discrimination) and CL-aware trigger optimization (to ensure persistence).
Empirical Validation: Extensive evaluation showing that PFBA induces severe fairness disparities that persist across CL rounds, evading standard defenses. Crucially, the authors reveal that certain anti-forgetting mechanisms (like Experience Replay) can inadvertently amplify the attack by reinforcing the very feature subspaces the backdoor exploits.

Experimental Results

The authors evaluated PFBA on four datasets (HAM10000, HIBA, CheXpert, CelebA) across three MLLM backbones (DeepSeek-VL, Qwen2.5-VL, LLaVA) and four CL strategies (LoRA, O-LoRA, SEFE, Experience Replay).

Persistence: On HAM10000, PFBA maintained a PBias (performance bias) above 27% after two CL rounds. In contrast, baseline methods (TrojFair, BadFair) collapsed to single-digit PBias (e.g., <10%) under the same conditions. On the DeepSeek backbone, PFBA amplified PBias to over 70%.
Stealthiness: The attack preserved high clean-data accuracy (>95%) with low aggregate bias (CBias < 1.5%), making it difficult to detect via standard performance audits.
Defense Evasion: Standard defenses, including Spectral Signatures and Fine-pruning, reduced but failed to eliminate the backdoor. Residual PBias persisted (e.g., 5.7% after aggressive pruning), exceeding typical fairness audit thresholds.
Generalizability: The attack generalized across different model architectures and scales. Furthermore, triggers optimized on one dataset (HAM10000) transferred to a different population (HIBA) with non-trivial PBias, though performance was lower than in-domain results.
Ablation: Removing either the Latent Space Reinforcement or the CL Simulation significantly degraded persistence or stealthiness, confirming the necessity of both components.

Significance and Claims

The paper claims that Continual Learning alone is insufficient to eliminate fairness-targeted backdoors. The study highlights a fundamental tension: mechanisms designed to prevent catastrophic forgetting (like Experience Replay) can inadvertently preserve and even amplify representation-level backdoors anchored to stable feature subspaces.

The authors position PFBA as a dual-use tool to expose security risks in third-party model checkpoints and motivate the development of representation-level defenses specifically targeting adversarial geometric structures in continually evolving models. The work underscores that fairness backdoors can remain undetected for extended periods, accumulating harm in high-stakes deployments.

Note: The paper explicitly states that its evaluation focuses on group-level discrimination in classification tasks under white-box injection and does not yet encompass open-ended generation or broader fairness notions.