🤖 AI

Inference-Only Prompt Projection for Safe Text-to-Image Generation with TV Guarantees

This paper introduces SPOT, an inference-time framework that selectively projects unsafe prompts toward a "tau safe set" using an LLM and a safeguard VLM to significantly reduce inappropriate image generation while preserving the behavior of benign prompts without retraining the diffusion model.

Original authors: Minhyuk Lee, Hyekyung Yoon, Myungjoo Kang

Published 2026-05-13

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Minhyuk Lee, Hyekyung Yoon, Myungjoo Kang

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a magical, frozen art machine (a Text-to-Image AI) that can draw anything you describe. It's incredibly talented, but sometimes, if you give it a tricky or dangerous prompt, it might accidentally draw something inappropriate.

The problem is: if you try to "fix" the machine itself to stop drawing bad things, you often accidentally break its ability to draw good things too. It's like trying to stop a chef from cooking spicy food by removing their knife; suddenly, they can't chop vegetables for a salad either.

This paper introduces a new method called SPOT (Selective Prompt Projection) that acts like a smart editor sitting before the machine, rather than trying to rebuild the machine itself.

Here is how it works, using simple analogies:

1. The "Frozen Machine" Rule

The authors decided not to touch the art machine at all. They kept it exactly as it was (frozen). Why? Because if you change the machine, you change its personality. They wanted to keep the machine's "good" behavior exactly the same, only stopping the "bad" behavior.

2. The "Safety Map" (The τ Safe Set)

Imagine a map where some areas are "Green Zones" (safe to draw) and others are "Red Zones" (unsafe).

The Goal: If you ask for something in a Green Zone, the editor leaves your request alone. The machine draws exactly what you asked for.
The Problem: If you ask for something in a Red Zone, the editor must intervene. But instead of just saying "No," it tries to find the nearest Green Zone that still looks like your original request.

3. The Two-Stage Detective Team

SPOT uses a two-step process to handle risky requests, acting like a security team with a cheap scanner and a strict inspector.

Stage 1: The Quick Scanner (The LLM)
When a risky prompt comes in, a fast, text-only AI (the LLM) acts as a scout. It quickly generates a few "rewritten" versions of your prompt. It asks: "If I change this word to that word, does it look safer?"
- It picks the version that is closest to your original idea but moves it out of the Red Zone and into the Green Zone.
- This is fast and cheap because it only looks at words, not pictures.
Stage 2: The Strict Inspector (The VLM)
Once the editor picks the best rewritten prompt, the actual art machine draws the picture. Then, a second AI (a Vision-Language Model) looks at the actual image that was created.
- It asks: "Is this picture actually safe?"
- If the picture is safe, it gets the green light.
- If the picture is still unsafe (maybe the machine misunderstood the new prompt), the system goes back to Stage 1, tries a different rewrite, and draws again.

4. The "Safety vs. Creativity" Trade-off

The paper makes a very important mathematical point: You cannot make an AI safer without changing its output slightly.
Think of it like a river. If you want to divert a dangerous flood (unsafe images) away from a village, you have to build a new channel. That new channel changes the water's path.

SPOT's trick: It only builds a new channel for the dangerous flood. If the water is already flowing safely (benign prompts), it leaves the river exactly where it is. This ensures that when you ask for a "cute cat," you still get a "cute cat," not a "cartoon cat" or a "black screen."

5. The Results

The authors tested this on four different sets of data and three different art machines.

Safety: It successfully reduced the number of inappropriate images by a lot (between 14% and 44% better than other methods).
Creativity: It kept the "good" images looking almost exactly the same as they would have without any safety filters.
Speed: Because the first stage (the text scanner) is so fast, the whole process is much quicker than methods that check every single image idea before drawing it.

Summary

SPOT is like a bouncer at a club who doesn't change the music or the DJ (the AI model). Instead, if someone tries to say something rude at the door, the bouncer gently suggests they rephrase it to something polite. If the rephrased sentence gets them in, they enter. If they keep trying to be rude, they don't get in. But if they were polite to begin with, they walk right in without being touched.

This ensures the club stays safe without ruining the experience for the good guests.

Technical Summary: SPOT – Selective Prompt Projection via Total Variation

Problem Statement
Text-to-Image (T2I) diffusion models enable high-quality open-ended synthesis but pose significant safety risks, as users can elicit harmful or policy-violating imagery. Existing safety alignment methods often suffer from a critical trade-off: global interventions (such as modifying generator weights or applying universal soft prompts) can perturb the model's behavior even on benign prompts, degrading image quality and semantic fidelity. Conversely, methods that verify generated images often incur high computational costs or fail to localize safety interventions to only the risky inputs. The paper identifies a fundamental tension: reducing expected unsafety below a reference level inherently requires a deviation in the prompt-conditioned image distribution. The challenge is to achieve safety without globally distorting the behavior of the frozen generator on benign inputs.

Methodology
The authors propose SPOT (Selective Prompt prOjecTion), an inference-time framework that approximates a "safe projection" without retraining the generator or learning category-specific rewriters. The methodology is grounded in a theoretical analysis using Total Variation (TV) distance.

Theoretical Foundation (SPAT): The paper formalizes the Safety–Prompt Alignment Tradeoff (SPAT). It proves that if unsafety is measured as a bounded expectation over generated images, reducing this expectation below a reference level (the frozen generator's behavior) necessitates a distributional deviation from that reference. Consequently, the goal is not to avoid deviation entirely, but to localize it strictly to prompts that require intervention.
$\tau$ -Safe Set and Projection: The authors define a $\tau$ -safe set of prompts, $C_{safe,\tau}$ , where the reference risk is at most a threshold $\tau$ . The intervention is formulated as projecting risky prompts (those outside $C_{safe,\tau}$ ) toward the nearest prompts within this set, while leaving already safe prompts unchanged.
Two-Stage Cascade Protocol: Since exact projection is intractable (as membership depends on expected image-level risk), SPOT approximates this via a two-stage cascade:
- Stage 1 (Prompt-Only Search): A Large Language Model (LLM) generates candidate rewrites for a given prompt. These candidates are ranked using a soft objective that balances semantic proximity to the original prompt (measured via sentence embedding distance) against a prompt-only safety score. This stage acts as a cheap router to identify promising candidates.
- Stage 2 (Image-Level Verification): The diffusion generator (kept frozen) samples an image from the best candidate prompt found in Stage 1. A safeguard Vision-Language Model (VLM) verifies the generated image against the same safety threshold $\tau$ .
- Retry Mechanism: If the image fails verification, the process repeats with new candidates up to a retry budget $R$ . If the budget is exhausted, the final attempt is returned.

Key Contributions

Reference-Based Safety Tradeoff: The paper formalizes SPAT, demonstrating that reducing population unsafety below a reference level requires prompt-conditioned distributional deviation. This provides a theoretical justification for selective intervention rather than global model modification.
Selective Prompt Projection: The authors define the $\tau$ -safe set and formulate safety intervention as projecting risky prompts toward this set while preserving safe prompts, effectively minimizing semantic drift on benign inputs.
Inference-Time LLM/VLM Cascade: SPOT instantiates this projection using an LLM for candidate ranking and a VLM for final image verification. This design separates cheap search from authoritative verification, avoiding the computational cost of exhaustive image generation during the search phase.
Strong Safety and Benign Preservation: The framework achieves significant reductions in inappropriate content while maintaining the behavior of the frozen generator on benign prompts.

Experimental Results
The authors evaluated SPOT across four datasets (CoProV2, I2P, Unsafe Diffusion, and COCO) and three diffusion backbones (SD1.5, SD2.1, SDXL).

Safety Performance: SPOT achieved relative inappropriate (IP) score reductions ranging from 14.2% to 44.4% compared to strong safety alignment baselines (e.g., AlignGuard, LatentGuard, SLD). On SD1.5, SPOT reduced IP to 0.04 on CoProV2, 0.06 on I2P, and 0.04 on UD, outperforming baselines that often showed higher IP on out-of-distribution datasets.
Benign Prompt Preservation: On the COCO dataset, SPOT preserved image quality (FID) and text-image alignment (CLIPScore) close to the unaligned reference. For instance, on SD1.5, SPOT achieved an FID of 32.46 and CLIPScore of 33.36, nearly identical to the unaligned baseline (32.34/33.42), whereas aggressive baselines like UCE and PNO showed significant degradation.
Selectivity: SPOT left 97.70% of benign COCO prompts unchanged, substantially higher than baselines like VALOR (92.4%) and IPR (54.9%), demonstrating its ability to avoid unnecessary edits.
Efficiency: The two-stage cascade provided a 4.3× to 5.1× speedup compared to VLM-only search methods. SPOT was 22.4% faster than VALOR and 61.1% faster than IPR while achieving lower IP scores.
Robustness: The method demonstrated robustness against four text-based adversarial prompt attacks (MMA, Ring-A-Bell, SneakyPrompt, P4D), with total IP scores rising only marginally (from 0.04 to 0.06) under attack.

Significance and Claims
The paper claims that SPOT offers a principled approach to safety alignment by leveraging the theoretical insight that safety gains require distributional deviation, but that this deviation should be confined to risky prompts. By keeping the generator frozen and using a selective projection mechanism, SPOT avoids the "semantic drift" and performance degradation often associated with global model editing or aggressive suppression. The authors position SPOT as a practical, inference-time solution that allows safety levels to be adjusted via a threshold ( $\tau$ ) without retraining, making it adaptable to different safety policies and datasets. The work emphasizes that safety does not require compromising the utility of benign prompts, provided the intervention is selective and verified at the image level.