BBQ-to-Image: Numeric Bounding Box and Qolor Control in Large-Scale Text-to-Image Models

This paper introduces BBQ, a large-scale text-to-image model that achieves precise spatial and chromatic control by directly conditioning on numeric bounding boxes and RGB triplets within a unified structured-text framework, eliminating the need for architectural changes or iterative prompting.

The Gist

Imagine you are an artist trying to paint a picture based on a friend's description.

The Old Way (Current AI Models):
Your friend says, "Put a red dog in the bottom right corner."
You paint it. But the dog is a bit too far left, and the red is more like a pinkish-orange. You ask your friend to "move it a bit more right" and "make it darker red." They say, "Okay, but not too dark." You repaint. They say, "Now it's too dark."
This is the current state of AI image generation. It's great at understanding ideas and vibes, but it's terrible at following exact numbers. It's like trying to hit a bullseye while wearing foggy glasses.

The New Way (BBQ):
Enter BBQ (Bounding-box and Qolor control). The authors of this paper realized that professional designers don't want to guess; they want precision. They want to say, "Put the dog exactly at coordinates (70, 80) and make its collar exactly RGB (255, 0, 0)."

Here is how BBQ works, broken down into simple concepts:

1. The "Recipe" Analogy

Most AI models are like chefs who read a recipe written in poetry: "Add a pinch of salt to the savory, golden soup." The result is usually good, but the salt might be too much or too little.

BBQ is like a chef who reads a recipe written in a spreadsheet: "Add exactly 3.5 grams of salt. Place the bowl at 12 inches from the left edge. Set the oven to exactly 375°F."
Because the AI is trained on these "spreadsheets" (structured data with exact numbers), it knows exactly where to put things and what color they should be.

2. How They Taught the AI (The "Training" Part)

You might think, "But AI models are huge and complex; you can't just tell them to use numbers."
The authors didn't change the AI's brain (its architecture). Instead, they changed the language it learned.

• They took millions of images.
• They used other smart tools to measure the exact location of every object (like a fire hydrant or a cat) and the exact color of every pixel.
• They wrote these measurements into the "captions" (the text instructions) the AI reads.
• They taught the AI: "When you see the numbers (10, 20, 50, 80), you must draw the object exactly there."

It's like teaching a child to draw by showing them a grid. Instead of saying "draw a house in the middle," you say, "draw the house starting at square 3, column 4."

3. The "Magic Bridge" (The User Interface)

Here is the tricky part: Humans aren't good at typing coordinates like (x: 45.2, y: 88.1). We just want to say, "Put a dog there."
So, the paper introduces a Bridge (a second, smaller AI).

1. You type: "A dog playing fetch in the park."
2. The Bridge translates your simple sentence into the "BBQ language": "Dog at (10, 10, 50, 50), Color (Red, Green, Blue)."
3. BBQ draws the picture using those exact numbers.

If you want to move the dog, you don't have to re-type the whole prompt. You just drag the dog in the picture (or change the numbers), and the Bridge updates the instructions. BBQ then redraws the scene, moving only the dog, leaving the trees and the sky exactly where they were.

4. Why This is a Big Deal (The "Disentanglement")

In normal AI, if you ask to "move the dog to the left," the AI might get confused and change the dog's breed, the time of day, or the background.
BBQ is disentangled. This means it treats the "where" and the "what" as separate knobs.

• Knob A (Location): Turn this, and only the position changes.
• Knob B (Color): Turn this, and only the color changes.
• Knob C (The Dog): Turn this, and the dog changes, but stays in the same spot.

It's like editing a photo in Photoshop where you can move a layer without affecting the layers underneath.

Summary

BBQ is a new super-power for AI art generators. It stops the AI from guessing and starts it from calculating.

• Before: "Make the shirt red." (AI: Maybe this shade of red? Or that one?)
• Now: "Make the shirt RGB(255, 0, 0) and put it at box (10, 10, 50, 50)." (AI: Done. Exactly.)

This allows designers and regular users to have total control over their images, turning AI from a "magic wand" that sometimes works into a "precision tool" that always does exactly what you ask.

Technical Summary

1. Problem Statement

While recent text-to-image (T2I) models have achieved high realism and fine-grained control through long, structured captions (e.g., FIBO, FLUX.2), a fundamental parametric gap remains.

• Descriptive vs. Deterministic: Current models rely on subjective linguistic descriptors (e.g., "crimson," "top-right"), which are inherently imprecise. Professional workflows, however, require deterministic, pixel-accurate control over object location, size, and exact color values (RGB).
• Limitations of Existing Solutions: Previous approaches to spatial or color control often require:
  • Complex architectural modifications (e.g., specialized grounding tokens).
  • Auxiliary conditioning mechanisms (e.g., ControlNet).
  • Inference-time optimization or training-free heuristics that compromise efficiency or consistency.
• The Goal: To enable large-scale T2I models to natively accept and execute explicit numeric parameters (bounding box coordinates and RGB triplets) within a unified text-based framework, without altering the model architecture.

2. Methodology

The authors introduce BBQ (Bounding Box and Qolor), a framework that achieves precise parametric control through data augmentation and a structured intermediate language, rather than architectural changes.

A. Core Concept: Structured Parametric Conditioning

BBQ conditions the generation process on a structured text prompt that includes:

1. Numeric Bounding Boxes: Defined as normalized coordinates $(x_0, y_0, x_1, x_2) \in (0, 1)^4$ representing the top-left and bottom-right corners.
2. Explicit RGB Colors: Defined as integer triplets $c \in [0, 255]^3$.
   These numeric values replace semantic descriptions (e.g., "red shirt" becomes "shirt at (x,y) with RGB(255,0,0)").

B. Training Data Augmentation

To train the model on these parameters without new architectures:

• Base Model: The authors start with the FIBO backbone (an 8B-parameter model trained on long structured JSON captions).
• Data Generation: They augment existing structured captions with precise numeric data:
  • Bounding Boxes: Extracted using Grounded SAM2.
  • Depth: Estimated using Depth Anything V2.
  • Colors: Dominant object colors extracted using Pylette.
• Dataset Scale: The model is fine-tuned on 25 million images paired with these enriched, parametric captions.
• Training Objective: Standard flow-matching formulation (no new loss functions or architectural heads). The model learns to map the numeric tokens in the text directly to spatial and chromatic features in the latent space.

C. The Parametric Bridge (Inference)

Since users cannot manually write complex JSON with exact coordinates and RGB values, BBQ employs a Vision-Language Model (VLM) as an inference-time bridge:

• Model: A fine-tuned Qwen-3 VL 4B (or capable models like Gemini 2.5).
• Function: It translates short natural-language prompts into the full structured parametric JSON required by BBQ.
• Modes:
  1. Generate: Expands a brief prompt into a full parametric JSON.
  2. Refine: Edits specific fields (e.g., moving a box or changing a color) in an existing JSON while maintaining global coherence.
  3. Inspire: Extracts parametric descriptions from a reference image.

3. Key Contributions

1. Architecture-Free Parametric Control: Demonstrates that precise control over location and color can be achieved purely through large-scale training on enriched data, eliminating the need for specialized tokens, adapters, or architectural changes.
2. Native Disentanglement: The model exhibits "native disentanglement," meaning modifying a specific numeric parameter (e.g., moving a bounding box) updates only that element while preserving the rest of the scene, even without observing the original image during re-generation.
3. Unified Framework: Unifies spatial (bounding box) and chromatic (RGB) control within a single text-based conditioning schema, bridging the gap between natural language intent and professional-grade numeric precision.
4. Intuitive Interaction: Enables user interfaces that mimic professional tools (drag-and-drop objects, color pickers) by translating user actions into structured parameter updates.

4. Experimental Results

The paper evaluates BBQ against state-of-the-art baselines (FIBO, Flux.2 Pro, Nano Banana Pro) and specialized grounding models (GLIGEN, InstanceDiffusion).

• Text-as-a-Bottleneck Reconstruction (TaBR):
  • BBQ outperforms all baselines in reconstructing images from captions, with a 93.3% win rate against Flux.2 Pro and 76.1% against FIBO. This proves that adding numeric parameters does not degrade expressiveness or global coherence.
• Bounding Box Accuracy:
  • Evaluated on COCO and LVIS datasets using YOLO and ViTDet detectors.
  • BBQ achieves an AP of 28.6 on COCO, significantly outperforming general T2I models (Flux.2: 3.5, Nano Banana: 5.0) and the dedicated grounding model GLIGEN (19.6).
  • While trailing the specialized InstanceDiffusion (38.8), BBQ achieves this without architectural modifications, offering a strong trade-off between general fidelity and spatial control.
• Color Fidelity:
  • Evaluated using CIELab clustering and $\Delta E_{00}$ (perceptual difference) and $a-b$ chromaticity distance.
  • BBQ achieves the lowest error rates in $a-b$ distance (mean 7.16 vs. ~10.0+ for others) for both $K=5$ and $K=8$ clusters, indicating superior control over hue and saturation independent of lighting.

5. Significance and Impact

• Paradigm Shift: BBQ proposes a new paradigm where user intent is first translated into an intermediate structured language (containing numeric parameters) before being rendered by the generative model. This separates the "intent translation" (VLM) from the "rendering" (Flow-based Transformer).
• Professional Viability: By replacing ambiguous prompting with precise, familiar controls (dragging, color picking), BBQ makes generative AI viable for professional design workflows that require deterministic outcomes.
• Scalability: The approach suggests that future models can integrate additional precise attributes (e.g., exact lighting angles, material properties) simply by enriching the training data, without the need for complex, model-specific engineering.

In summary, BBQ demonstrates that large-scale text-to-image models can natively understand and execute numeric constraints, effectively closing the gap between creative text prompting and deterministic, professional image synthesis.