GASS: Geometry-Aware Spherical Sampling for Disentangled Diversity Enhancement in Text-to-Image Generation

This paper introduces Geometry-Aware Spherical Sampling (GASS), a method that enhances diversity in text-to-image generation by explicitly controlling prompt-dependent and prompt-independent variations through geometric decomposition of CLIP embeddings, achieving disentangled diversity with minimal impact on image fidelity and semantic alignment.

The Gist

Imagine you have a magical art machine (a Text-to-Image AI) that you ask to draw "a black car." You press the button, and it gives you a picture. You press it again, and it gives you another picture. But here's the problem: every time you press the button, the car looks almost exactly the same. It's always the same angle, in the same parking lot, with the same lighting. The machine is stuck in a rut.

This paper introduces a new trick called GASS (Geometry-Aware Spherical Sampling) to fix this. It helps the machine generate a much wider variety of pictures from the same prompt, without making the pictures look weird or breaking the rules of the prompt.

Here is how it works, explained with simple analogies:

1. The Problem: The "Stuck" Artist

Current AI models are great at following instructions, but they are bad at being creative when you don't give them extra instructions. If you ask for "a black car," the AI knows exactly what a car is (the prompt-dependent part), but it gets lazy about everything else (the prompt-independent part). It defaults to the same background, the same lighting, and the same style every time.

2. The Solution: The "Two-Track" Map

The authors realized that the AI's "brain" (its internal math space) can be split into two distinct directions, like a map with two axes:

• Track A (The Prompt): This is the part of the image that must follow your text. If you say "black car," this track ensures the car is black.
• Track B (The Freedom): This is the part of the image that the text didn't specify. This is where the background, the time of day, the camera angle, and the artistic style live.

Most previous methods tried to shake the whole map to get variety, which often made the car look weird or changed the color from black to blue. GASS is smarter: it only shakes Track B.

3. The Magic Trick: "Expanding the Bubble"

Imagine the AI's possible outputs are all floating inside a giant, invisible bubble (a sphere).

• The Old Way: To get variety, people would just push the images randomly inside the bubble. Sometimes they pushed them so hard they hit the wall and broke the image (making it look bad).
• The GASS Way: The authors realized that the AI was clustering all its images in one tiny corner of the bubble. GASS acts like a gentle hand that pushes the images apart along two specific lines:
  1. Along the Prompt Line: Making sure the "black car" idea is explored from different angles (front view, side view, top view).
  2. Along the Freedom Line: Pushing the images into new areas of the bubble to explore new backgrounds (a beach, a city street, a garage) and lighting (sunny, rainy, neon).

4. How It Actually Works (The "Tweak")

The process happens while the AI is "dreaming" up the picture, step-by-step:

1. The Guess: The AI guesses what the final picture should look like at a specific moment.
2. The Check: The system looks at this guess and asks, "Is this too similar to the other guesses we made?"
3. The Nudge: If the guesses are too crowded, GASS gives them a tiny, calculated nudge. It stretches the "freedom" part of the image (changing the background) and the "angle" part of the image, but it carefully holds onto the "black car" part so the meaning doesn't get lost.
4. The Result: The AI continues its dream, but now it's walking a wider path, resulting in a batch of images that are all unique, yet all still clearly "black cars."

Why Is This Cool?

• No More Boring Backgrounds: In the paper's examples, when asked for "a black car," other methods made cars that looked the same but with blurry, undefined backgrounds. GASS made cars with distinct, detailed backgrounds (a garage, a mountain road, a city).
• It's Safe: It doesn't ruin the image quality. The cars still look like cars, and the text still matches.
• It's Flexible: You can tell the AI to focus only on changing the background, or only on changing the angle, or both. It's like having a remote control for creativity.

In short: GASS is like a tour guide for an AI artist. Instead of letting the artist wander aimlessly (which leads to bad art) or stay in one spot (which leads to boring art), GASS gently guides the artist to explore the whole neighborhood, ensuring every photo is a unique masterpiece while still sticking to the main instruction.

Technical Summary

1. Problem Statement

Modern Text-to-Image (T2I) generative models (e.g., Diffusion, Flow-based) have achieved high semantic alignment and image fidelity. However, they suffer from a critical limitation: lack of diversity when generating multiple images from a single fixed text prompt.

• Consequences: This restricts user creative control and risks amplifying societal biases by reinforcing narrow visual stereotypes (e.g., specific genders or ethnicities for certain roles).
• Limitations of Existing Methods: Current diversity enhancement techniques primarily rely on entropy maximization (e.g., Vendi Score) to increase sample dissimilarity. These approaches treat diversity as a monolithic concept, failing to distinguish between:
  • Prompt-dependent variation: Semantic changes aligned with the text (e.g., object pose, viewing angle).
  • Prompt-independent variation: Unspecified attributes (e.g., background, lighting, style).
  • Existing decomposition methods (like Scendi) often rely on covariance matrices that degenerate in fixed-prompt settings, limiting their applicability.

2. Methodology: GASS (Geometry-Aware Spherical Sampling)

The authors propose GASS, a framework that enhances diversity by explicitly controlling geometric variations within the CLIP embedding hypersphere. The method operates in three main stages:

A. Geometric Decomposition of Diversity

The core insight is to decompose the variance of generated image embeddings ($e_i$) relative to the text embedding ($e_t$) into two orthogonal components:

1. Prompt-Dependent Axis ($e_t$): Captures semantic alignment with the prompt.
2. Prompt-Independent Axis ($u_{ind}$): A unit vector identified as the dominant direction of residual variance orthogonal to $e_t$. This captures attributes like backgrounds and styles that the prompt does not specify.

The Dominant Residual Basis Identification (Algorithm 1) involves:

• Generating candidate vectors orthogonal to $e_t$ via Gram-Schmidt orthogonalization.
• Selecting the vector ($u_{ind}$) that maximizes the mean absolute projection magnitude of the image batch, effectively finding the direction of greatest "unspecified" variation.

B. Spherical Spread Score (SPP)

To quantify diversity, the authors define a Spherical Spread Score (SPP):
$$SPP = D_{dep} + D_{ind}$$
Where $D_{dep}$ and $D_{ind}$ are the projection spreads (max - min) of the image batch along $e_t$ and $u_{ind}$, respectively. Experiments show that real-world images have significantly higher SPP scores than synthetic generations, validating this metric.

C. The GASS Sampling Process

GASS intervenes during the inference (sampling) steps of a frozen T2I model (e.g., Stable Diffusion):

1. Latent Expansion: At a specific sampling step $t$, the model predicts a clean image $\hat{x}_{0|t}$. This is encoded into CLIP space to get $e_i$.
2. Projection Modulation: The embedding is perturbed by adding expansion shifts ($\delta$) along both $e_t$ and $u_{ind}$ axes:
   $$\tilde{e}_i = (e_i^T e_t + \delta_{dep})e_t + (e_i^T u_{ind} + \delta_{ind})u_{ind} + r_i$$
   where $r_i$ is the residual. The vector is then re-normalized to stay on the unit hypersphere.
3. Gradient-Based Optimization: Since CLIP lacks a decoder, the method uses the frozen CLIP image encoder to guide the generation. A loss function ($L_{SPP}$) measures the alignment between the current image embedding and the expanded target $\tilde{e}_i$.
   $$L_{SPP} = \sum (1 - E_I(\hat{x}_{0|t})^T \tilde{e}_i)$$
   The estimated clean image $\hat{x}_{0|t}$ is updated via gradient descent to minimize this loss, effectively steering the generation trajectory toward the expanded geometric spread.

3. Key Contributions

1. Geometric Framework for Disentanglement: Introduced a novel method to disentangle and quantify prompt-dependent and prompt-independent diversity sources within the CLIP hypersphere using orthogonal projections.
2. GASS Algorithm: Proposed a sampling-based guidance method that explicitly expands the geometric spread of embeddings along identified orthogonal axes without modifying the text prompt or the generative backbone.
3. Controllability: Demonstrated the ability to selectively enhance diversity in specific dimensions (e.g., varying backgrounds while keeping object poses fixed, or vice versa) by modulating expansion ranges ($r_{dep}, r_{ind}$).
4. Theoretical Guarantee: Proved that the GASS guidance increases the expected hypervolume of the point set in the embedding space.

4. Experimental Results

The method was evaluated on ImageNet and DrawBench using diverse backbones (SD2.1, SD3-M, U-Net, DiT, Diffusion, Flow).

• Diversity Performance: GASS achieved state-of-the-art (SOTA) performance on diversity metrics (Vendi Score, Coverage) compared to baselines like Particle Guidance (PG), CADS, IG, and SPELL.
• Quality and Alignment: Unlike many diversity methods that degrade image quality, GASS maintained competitive ClipScore (semantic alignment) and ImageReward (perceptual quality), with minimal degradation.
• Qualitative Improvements: Visual comparisons showed that GASS generates significantly richer backgrounds and styles (prompt-independent) compared to baselines, which often produce smoothed or ambiguous backgrounds.
• Ablation Studies:
  • Re-normalization: Crucial for maintaining high image quality; removing it caused drops in quality metrics.
  • Expansion Range: Optimal trade-offs found at expansion ranges of 0.02.
  • Sparsity: The method is computationally efficient, requiring intervention only for a subset of sampling steps (e.g., 15-20 steps).

5. Significance and Impact

• Societal Impact: By enabling diverse generation from a single prompt, GASS helps mitigate the reinforcement of societal biases and stereotypes inherent in current T2I models.
• Technical Advancement: It shifts the paradigm from "blind" entropy maximization to structured, geometric control. It is the first method to explicitly introduce meaningful background diversity without prompt engineering.
• Generalizability: The approach is model-agnostic, working effectively across different architectures (U-Net vs. DiT) and generation paradigms (Diffusion vs. Flow).
• Future Directions: The geometric decomposition framework opens avenues for controlling other factors of variation beyond text, such as layout or reference images, offering finer-grained control over generative processes.