Attention to Neural Plagiarism: Diffusion Models Can Plagiarize Your Copyrighted Images!

This paper reveals that diffusion models can plagiarize copyrighted images by employing a purely gradient-based "anchors and shims" method to perturb cross-attention mechanisms, thereby bypassing both visible and invisible copyright protections without requiring additional training.

The Gist

The Big Idea: The "Neural Photocopier" That Erases Fingerprints

Imagine you have a priceless, original painting. To protect it, you've painted a tiny, invisible fingerprint on the back (a watermark) and maybe even signed the front with a visible signature. You think you're safe.

Now, imagine a super-smart robot (a Diffusion Model) that has seen millions of paintings. This paper reveals a scary new trick: This robot can look at your protected painting, copy it perfectly, and then magically erase your invisible fingerprint and change your signature so it looks like a completely different, original piece of art.

The authors call this "Neural Plagiarism." It's not just copying; it's stealing the idea of the image while scrubbing away the proof that you own it.

---

How Does the Robot Do It? (The "Shim" and "Anchor" Trick)

The researchers discovered a way to hack the robot's brain without needing to retrain it. They used a method they call "Anchors and Shims."

1. The Anchor (The Blueprint)

First, the robot takes your copyrighted image and breaks it down into a series of mathematical "snapshots" (called latents). Think of these snapshots as the Anchors. They are the perfect, unchangeable blueprint of your image as the robot sees it.

2. The Shim (The Wedge)

Here is the clever part. In real life, if you want to fix a wobbly door, you slide a thin piece of wood (a shim) into the gap to adjust it.

• The researchers slide these "shims" (tiny mathematical nudges) into the robot's blueprint.
• They don't just nudge the picture; they nudge the math behind the picture.
• By pushing the blueprint slightly away from the original "Anchor," they force the robot to generate a new image that looks almost exactly like yours, but the invisible fingerprint is now broken or changed.

The Two Types of "Theft"

The paper shows this method can do two dangerous things:

1. The "Forgery" Attack (Erasing the Evidence)

• The Goal: Make a copy that looks like the original but has no watermark.
• The Analogy: Imagine a forger who takes your signed painting, paints over your signature with a fake one, and then claims, "This is my original art, I made it from scratch!"
• The Result: The robot creates a replica that is so similar to the original that a human can't tell the difference, but the invisible digital watermark is gone. The copyright detector says, "I see no watermark," so it thinks the image is free to use.

2. The "Ambiguity" Attack (The He Said/She Said)

• The Goal: Make a copy that has two different watermarks on it.
• The Analogy: Imagine a forger takes your painting, erases your signature, and writes their name on it. But, they also leave a faint, ghostly trace of your name. Now, if you go to court, the forger says, "See? It has my name!" and you say, "No, it has mine!"
• The Result: The robot creates an image that triggers both your watermark and a new, fake watermark. This creates a legal nightmare where no one knows who actually owns the image.

Why Is This a Big Deal?

Usually, people think, "If I watermark my image, it's safe." Or, "If I use legal laws, the AI won't copy me."

This paper proves that neither is true against this specific type of attack.

• Invisible Watermarks: The robot can scrub them out.
• Visible Trademarks/Signatures: The robot can change the style of the image (e.g., turning a long dress into a short skirt, or changing a face shape) just enough to break the legal definition of "copying," while keeping the image recognizable.
• No Training Needed: The scary part is that the attacker doesn't need to teach the robot anything new. They just use a clever math trick (gradient search) to find the "shims" that work.

The Takeaway

The authors aren't trying to teach people how to steal; they are sounding an alarm. They are saying: "Our current locks (watermarks) don't work on this new kind of thief (Diffusion Models)."

They built this "Neural Plagiarism" tool to show the world that we need better security measures immediately. If we don't, artists, photographers, and brands could lose control of their work to AI models that can effortlessly replicate and sanitize their content.

In short: The paper shows that AI can now be a master thief that not only copies your house but also changes the locks and the address so you can't prove you live there anymore.

Technical Summary

1. Problem Statement: Neural Plagiarism

The paper identifies a critical vulnerability in modern generative AI, specifically Diffusion Models, termed "Neural Plagiarism."

• The Threat: Diffusion models, trained on vast datasets, can memorize and reproduce copyrighted content (images, artworks, signatures, trademarks) even when protected by advanced copyright mechanisms.
• The Gap: Existing legal frameworks and technical defenses (such as visible watermarks, signatures, and invisible digital watermarks like Tree-Ring or Stable Signature) are insufficient. Attackers can use generative models to create replicas that bypass these protections or create "ownership ambiguity."
• Attack Goals: The authors define two primary attack vectors:
  1. Forgery Attack: Replicating a copyrighted image while completely removing the embedded watermark, causing verification to fail ($V(x^*) \neq w$).
  2. Ambiguity Attack: Removing the original watermark and embedding a new watermark ($w^*$), creating competing claims of ownership and confusing the verification process ($V(x^*) = w^*$ and $V(x_w) = w$).

2. Methodology: Anchors and Shims

The proposed method is a purely gradient-based search that requires no additional training or fine-tuning of the diffusion model. It operates in the latent space of the diffusion model.

Core Concepts

• Anchors: The authors first invert the target copyrighted image ($x_w$) through the diffusion process to obtain a sequence of "anchor" latents ($\{\hat{x}_t\}$). These anchors represent the trajectory that generates the original image.
• Shims: Instead of optimizing the entire latent trajectory (which is memory-intensive), the method introduces small perturbations called "shims" ($\delta_t$) at specific timesteps. These shims act like physical shims in door installation, adjusting the spacing to deviate the generated image from the anchor without breaking the overall structure.

Technical Mechanism

The attack optimizes the perturbation $\delta_t$ by minimizing a composite loss function at selected timesteps:

1. Norm Loss ($L_{norm}$): Ensures the perturbation $\delta_t$ is large enough to disrupt the watermark (enforcing a minimum distance from the anchor).
2. Semantic Loss ($L_{semantic}$): Uses cross-attention mechanisms in the UNet. It perturbs the text embedding (specifically the empty string embedding $e_\emptyset$) to find alternative Query, Key, and Value vectors that maintain semantic consistency with the original prompt while altering the visual output.
3. Alignment Loss ($L_{align}$): Ensures the perturbed latent at the next timestep remains close to the corresponding anchor latent, preventing the image from becoming noise.

Optimization Strategy:

• The process iterates backward from a selected timestep $K$ to 1.
• Early Start (Large $K$): Injecting shims early induces large semantic changes (e.g., changing a bus color or hairstyle), useful for altering copyrighted content significantly.
• Late Start (Small $K$): Injecting shims late induces minor, localized changes, effective for removing invisible watermarks while preserving high image fidelity.

3. Key Contributions

1. General Neural Plagiarism Pipeline: A novel framework demonstrating that diffusion models can replicate copyrighted data while evading both visible and invisible protections.
2. Anchors and Shims Framework: A memory-efficient, gradient-based search method that decouples the latent chain, allowing for targeted perturbations without the massive GPU memory overhead of full trajectory optimization.
3. Attention-Based Perturbation: A technique that manipulates the cross-attention mechanism to find alternative semantic representations, enabling "coarse-to-fine" exploration of the semantic space.
4. Comprehensive Threat Analysis: The paper provides empirical evidence of both Forgery (removing watermarks) and Ambiguity (creating conflicting ownership claims) attacks across various datasets and watermarking techniques.

4. Experimental Results

The authors evaluated their method on MS-COCO and real-world copyrighted images (Disney characters, famous artworks, signatures, and portraits like Elon Musk).

• Invisible Watermark Removal:

  • Post-hoc Watermarks (DwtDctSvd, RivaGAN): The method achieved ~50% Bit Accuracy (BA) on watermark detection (effectively random guessing), significantly outperforming baselines like Regen and Rinse.
  • Latent Watermarks (Stable Signature): Successfully bypassed the watermark, reducing detection rates to near zero.
  • Tree-Ring Watermark: While the method could not remove the Tree-Ring pattern (due to its robustness in Fourier space), it successfully created Ambiguity Attacks by embedding a competing watermark, confusing the ownership verification.

• Copyrighted Image Replication:

  • The method generated replicas of copyrighted characters (e.g., Elsa from Frozen) and real people (Elon Musk) that visually resembled the originals but featured distinct semantic changes (e.g., different hairstyles, facial structures) to bypass copyright filters.
  • In the "Elon100" experiment, 100 distinct replicas were generated, none of which were definitively identified as the original subject by an AI arbiter, yet all retained the core identity.

• Efficiency: The method requires no model training and operates with significantly lower memory consumption than full latent optimization approaches.

5. Significance and Implications

• Vulnerability Exposure: The paper highlights that current technical defenses (watermarking) are fragile against generative AI. Even "robust" watermarks can be circumvented or rendered ambiguous.
• Legal and Ethical Challenges: The ability to create "Ambiguity Attacks" poses a severe threat to copyright law. If an attacker can generate a replica with a different valid watermark, it becomes nearly impossible to prove original ownership, undermining the legal basis of copyright.
• Call to Action: The authors argue that the current landscape of AI-generated content requires urgent countermeasures. The existence of "Neural Plagiarism" suggests that relying solely on watermarking is insufficient; future protections must address the fundamental memorization and replication capabilities of diffusion models.

Conclusion: The paper demonstrates that diffusion models can be weaponized to plagiarize copyrighted works by manipulating latent spaces via "anchors and shims," effectively bypassing existing watermarking technologies and creating significant legal ambiguities regarding data ownership.