💻 computer science

Beyond Text Prompts: Precise Concept Erasure through Text-Image Collaboration

The paper introduces TICoE, a text-image collaborative framework that achieves precise concept erasure in text-to-image models while preserving content fidelity through a continuous convex concept manifold and hierarchical visual representation learning.

Original authors: Jun Li, Lizhi Xiong, Ziqiang Li, Weiwei Jiang, Zhangjie Fu, Yong Li, Guo-Sen Xie

Published 2026-04-20

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Jun Li, Lizhi Xiong, Ziqiang Li, Weiwei Jiang, Zhangjie Fu, Yong Li, Guo-Sen Xie

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a magical artist named AI. This AI is incredibly talented; it can paint anything you describe, from a sunset to a futuristic city. However, because the AI learned from the entire internet, it also learned some "bad habits." If you ask it to draw a "gun," it might draw one even if you didn't want it to. If you ask for a "Van Gogh painting," it might copy his style too perfectly, which could be a copyright issue.

The problem is: How do you teach the AI to forget these specific bad habits without making it forget how to draw anything?

The Old Way: The "Blunt Hammer"

Previous methods tried to fix this in two ways, but both had flaws:

The Text-Only Approach: Imagine you tell the AI, "Don't draw guns." The AI hears you, but it's a bit stubborn. If you say "firearm" or "pistol" instead, it might still draw a gun because it didn't understand the full idea of a gun, just the specific word "gun." It's like telling a child, "Don't eat apples," but they still eat pears because you didn't say "fruit."
The Image-Guided Approach: Imagine you show the AI a picture of a gun and say, "Forget this." The AI gets the message, but it gets too confused. It might think, "Oh, I need to forget anything that looks like a gun." So, it accidentally stops drawing cameras, microphones, or toys because they have similar shapes (long, cylindrical, held in a hand). It's like a chef who, after being told to stop using salt, decides to stop using any white powder, ruining the sugar and flour too.

The New Solution: TICoE (The "Smart Librarian")

The paper introduces a new method called TICoE (Text-Image Collaborative Erasure). Think of TICoE as a Smart Librarian who is very good at organizing the AI's memory.

Here is how it works, using two clever tricks:

1. The "Continuous Concept Map" (The Text Trick)

Instead of just saying "gun," the Smart Librarian creates a 3D map of the concept.

How it works: It asks a super-smart helper (an AI language model) to generate hundreds of different ways to describe a gun: "revolver," "pistol," "firearm," "tactical weapon," "old musket," etc.
The Magic: It blends all these descriptions together into a smooth, continuous cloud of meaning. This ensures that no matter how you try to trick the AI (by using a fancy word or a weird phrase), the AI knows exactly what "gun" means and can remove it completely. It's like erasing the entire concept of a gun from the library, not just the book titled "Gun."

2. The "Layered Visual Glasses" (The Image Trick)

This is the most important part. The Smart Librarian puts on special glasses that let it see an image in layers.

How it works: When you show the AI a picture of a gun, the glasses break the image down into different sizes:
- Zoomed Out: It sees the whole scene (a soldier holding a weapon).
- Zoomed In: It sees the details (the trigger, the barrel).
- Context: It sees what's around the gun (the soldier's hand, the background).
The Magic: The AI learns to say, "I will erase the gun part, but I will keep the soldier's hand and the background." It learns to distinguish between the target (the gun) and things that just look similar (like a camera or a microphone). It's like a surgeon who removes a tumor but leaves the healthy skin around it perfectly intact.

The Result: A Safer, Smarter Artist

By combining these two tricks, TICoE achieves something previous methods couldn't:

Precision: It removes the bad concept (like "guns" or "nudity") completely, even if you try to trick it with new words.
Fidelity: It doesn't break the rest of the artist's skills. You can still ask for a "camera," a "phone," or a "Van Gogh-style landscape" (without the specific artist's name), and the AI will draw them perfectly.

The "MCP" Score: The "Did You Break Anything?" Test

The authors also invented a new test called MCP (Morpho-Contextual Concept Preservation).

Imagine: You tell the AI to forget "guns."
The Test: You then ask it to draw a "camera."
Old Methods: Might draw a broken camera or no camera at all because they got confused.
TICoE: Draws a perfect camera.
Why it matters: This proves the AI didn't just get "dumber"; it got smarter at knowing exactly what to forget and what to keep.

In a Nutshell

TICoE is like teaching a child to be more specific. Instead of saying "Don't touch anything red" (which stops them from touching apples and fire trucks), you teach them, "Don't touch the red fire extinguisher on the wall, but the red apple on the table is fine."

This makes AI safer, more controllable, and much more useful for creating art without accidentally generating harmful content or ruining unrelated ideas.

1. Problem Statement

Text-to-image generative models (e.g., Stable Diffusion) often inadvertently produce unsafe, sensitive, or undesirable content due to biases in their large-scale training datasets. While "concept erasure" (unlearning specific concepts like "guns" or "nudity") is a proposed solution, existing methods face a critical trade-off between erasure precision and content fidelity:

Text-Only Methods: Approaches like ESD, UCE, or attention optimization often fail to fully suppress concepts. They rely on fixed textual embeddings, which cannot capture the full semantic scope of a concept. Consequently, semantically related but lexically distinct prompts (e.g., "firearm" vs. "gun") can reactivate the erased concept.
Image-Assisted Methods: Recent attempts to use reference images improve coverage but introduce visual entanglement. Models often over-erase, suppressing semantically unrelated concepts that share visual patterns (e.g., erasing "guns" might inadvertently suppress "cameras" due to similar shapes).
Evaluation Gap: Current benchmarks focus on erasure strength but lack metrics for post-erasure usability, specifically failing to measure whether semantically distinct but visually similar concepts are preserved.

Goal: Develop a framework that precisely removes target concepts while preserving the model's ability to generate high-quality, semantically coherent, and visually related benign content.

2. Methodology: TICoE Framework

The authors propose TICoE (Text-Image Collaborative Erasing), a framework that jointly optimizes erasure precision and contextual fidelity through two core mechanisms:

A. Continuous Convex Concept Manifold (CCCM)

To address the limitations of discrete text prompts, TICoE constructs a continuous representation of the target concept.

Prompt Expansion: Using an LLM (GPT-5.0), a base concept keyword is expanded into a diverse set of semantically consistent prompts (e.g., "gothic church," "ancient stone church").
Convex Interpolation: Instead of using fixed embeddings, the model samples a textual embedding ( $e_c$ $e_{c}$ ) as a convex combination of the prompt bank embeddings.
- $e_c = \sum w_i e_i$ , where weights $w$ are drawn from a Dirichlet distribution.
Significance: This ensures the interpolated embedding remains within the semantic hull of the target concept, preventing "out-of-distribution" extrapolation. It creates a robust, continuous manifold that covers diverse linguistic expressions, making the erasure robust against adversarial or varied prompts.

B. Hierarchical Visual Representation Learning (HVRL)

To prevent over-erasure of visually similar but semantically distinct objects, TICoE encodes reference images into multi-scale features.

Multi-Scale Encoding: Reference images are encoded into latent features at multiple scales ( $s \in \{1.0, 0.75, 0.5\}$ ).
Transformer Fusion: These multi-scale tokens are processed through a stack of transformer encoder layers to capture hierarchical visual and contextual semantics.
Residual Fusion: The fused visual representation is added to the original latent space via a residual connection.
Significance: This allows the model to distinguish between the target concept and visually correlated entities. By learning causal associations rather than just visual correlations, the model can suppress the target (e.g., a gun) without suppressing visually similar benign objects (e.g., a camera).

C. Concept Erasure Loss

The framework jointly optimizes a trainable U-Net ( $\theta^*$ ) and the transformer encoder.

Classifier-Free Guidance (CFG): A reference target is constructed using the frozen pre-trained model with a negative guidance weight ( $\gamma$ ).
Objective: The loss function minimizes the difference between the trainable model's noise prediction and the reference target, effectively driving the model to suppress the target concept while maintaining the distribution of benign prompts.

3. Key Contributions

TICoE Framework: A novel text-image collaborative erasure method that combines a Continuous Convex Concept Manifold (for robust textual coverage) and Hierarchical Visual Representation Learning (for precise visual disambiguation).
Morpho-Contextual Concept Preservation (MCP): A new evaluation metric designed to measure usability. Unlike standard metrics, MCP explicitly evaluates whether semantically distinct but morphologically or contextually related concepts (e.g., "camera" when erasing "gun") remain intact after erasure.
State-of-the-Art Performance: Extensive experiments demonstrate that TICoE outperforms existing text-only and image-assisted methods in both erasure precision (lower ASR, UDA, P4D) and content fidelity (lower FID, higher CLIP/MCP).

4. Experimental Results

The authors evaluated TICoE on three categories: Nudity (safety), Style (Van Gogh), and Objects (Gun, Tench, Church).

Erasure Precision: TICoE achieved the lowest ASR (Attack Success Rate), UDA (Universal Adversarial Perturbation), and P4D (Prompt4Debugging) scores across all tasks. For example, on the "gun" task, TICoE achieved an ASR of 0.00 and UDA of 0.02, significantly outperforming competitors like ESD and Co-Erasing.
Fidelity & Preservation (MCP):
- TICoE maintained high generation quality (low FID, high CLIP).
- Crucially, on the MCP metric, TICoE showed superior preservation of related concepts. When erasing "gun," TICoE preserved "camera" with an MCP score of 92.06%, whereas Co-Erasing dropped to 39.68% (indicating severe over-erasure).
Generalization: The method proved robust across different Stable Diffusion backbones (v1.4, v1.5, v2.0) and successfully handled multi-concept erasure (e.g., erasing "church," "Van Gogh," and "cat" simultaneously) without mutual interference.
Fine-Grained Analysis: In nudity erasure, TICoE reduced failure cases in sensitive categories (e.g., "buttocks exposed") to near zero, outperforming methods that left residual activations.

5. Significance and Impact

Solving the Precision-Fidelity Trade-off: TICoE resolves the long-standing dilemma where improving erasure strength usually degrades image quality or causes collateral damage to unrelated concepts.
Robustness to Adversarial Attacks: By constructing a continuous concept manifold, the method is significantly more robust against "jailbreaking" prompts or adversarial perturbations that typically reactivate erased concepts in text-only models.
Responsible AI: The introduction of the MCP metric shifts the evaluation paradigm from "how much did we delete?" to "how well did we preserve the model's utility?" This is critical for deploying safe generative models in real-world applications where over-censorship is as problematic as under-censorship.
Practical Applicability: The framework provides a unified, controllable approach for "unlearning" specific concepts, making text-to-image generation safer and more reliable for sensitive domains.

In summary, TICoE represents a significant advancement in model editing by leveraging the complementary strengths of textual generalization and visual grounding to achieve faithful and precise concept erasure.