Easier Painting Than Thinking: Can Text-to-Image Models Set the Stage, but Not Direct the Play?

This paper introduces T2I-CoReBench, a comprehensive benchmark featuring 1,080 complex prompts and a 12-dimensional taxonomy to rigorously evaluate text-to-image models' composition and reasoning capabilities, revealing that while models struggle with high-density composition, their ability to perform implicit reasoning remains a critical bottleneck.

The Gist

Imagine you are hiring an artist to paint a picture based on a story you tell them. In the world of Artificial Intelligence, this is called Text-to-Image generation. You type a prompt like "a cat sitting on a red mat," and the AI tries to paint it.

For a long time, these AI artists were getting really good at the basics. But a new paper, published for the ICLR 2026 conference, asks a tough question: "Can these artists paint the scenery, but do they actually understand the story?"

The authors created a massive new test called T2I-COREBENCH to find out. Here is the breakdown in simple terms.

1. The Two Skills: "Painting the Stage" vs. "Directing the Play"

The paper argues that making a good image requires two very different skills:

• Composition (Painting the Stage): This is about putting the right objects in the right places. If you say "a red apple on a blue table," the AI needs to paint a red apple and a blue table.
  • The Analogy: This is like a stagehand setting up the props. They need to make sure the chair is there, the lamp is on the table, and the curtain is blue.
• Reasoning (Directing the Play): This is about understanding what happens or what is implied without you explicitly saying it. If you say "a ripe tomato is squeezed tightly in a fist," the AI needs to paint the tomato bursting with juice, even though you didn't write "juice."
  • The Analogy: This is the director telling the actors how to act. If the script says "the villain is angry," the actor needs to look angry, not just stand there. The AI needs to understand cause-and-effect, logic, and common sense.

2. The New Test: T2I-COREBENCH

Previous tests were like asking the artist to draw "a dog" or "a cat." Easy!
This new test is like asking them to draw a complex movie scene with a specific script.

• High Density: They don't just ask for one object. They ask for a kitchen with 20 different items, specific colors, and specific relationships (e.g., "the spoon is inside the pot, but the pot is under the table").
• The Checklist: To grade the AI fairly, the researchers didn't just look at the picture and say "looks good." They created a 13,500-question checklist.
  • Example: "Is the spoon inside the pot?" (Yes/No). "Is the pot under the table?" (Yes/No).
  • They used a super-smart AI (Gemini) to grade the answers based only on what is visible in the picture, ignoring what the AI thought it was drawing.

3. The Results: The "Stagehand" is Great, the "Director" is Lost

The paper tested 38 different AI models (from open-source ones to big tech giants like OpenAI and Google). Here is what they found:

• The Good News (Composition): The AI artists are getting much better at setting the stage. They can handle complex scenes with many objects. The gap between open-source models and expensive commercial ones is shrinking.
  • Metaphor: The stagehands are now very efficient at moving furniture around.
• The Bad News (Reasoning): The AI is terrible at understanding the story. Even the smartest models struggle with logic, cause-and-effect, and "what if" scenarios.
  • Metaphor: The director is asleep. If you tell the AI, "A car drives off a cliff," it might draw a car floating in the sky because it doesn't understand gravity. If you say, "A square wheel on a car," it might draw a round wheel because it thinks that's what a car should look like, ignoring your specific rule.

4. Why "Rewriting the Script" Doesn't Fully Fix It

The researchers tried a trick: they asked a super-smart text AI (like a human editor) to rewrite the prompt to be more explicit before giving it to the image generator.

• Example: Instead of "A car with square wheels," the editor writes, "A car where every single wheel is a perfect square, and the tires are square too."
• The Result: It helped a little, but not enough. The AI still struggled to break its own habits. It's like telling a stubborn actor, "I know you usually cry when sad, but for this scene, you must laugh." They often forget and just cry.

5. The Big Takeaway

The paper concludes that Text-to-Image AI is currently "Easier Painting Than Thinking."

We have built machines that are incredible at mimicking visual styles and arranging objects. But they are not yet "thinking" machines. They are following a checklist of visual patterns rather than truly understanding the logic of the world.

In short: We have taught the AI how to paint a picture of a kitchen. But we haven't taught it how to understand that if you drop an egg, it breaks. Until the AI learns to "direct the play" and not just "set the stage," it will keep making logical mistakes in complex situations.

Technical Summary

1. Problem Statement

Current Text-to-Image (T2I) generation models have made significant strides in generating high-quality images from text. However, existing evaluation benchmarks suffer from two critical limitations:

• Lack of Comprehensiveness: Most benchmarks evaluate either composition (generating explicit visual elements) or reasoning (inferring implicit elements) in isolation, using heuristic taxonomies that fail to cover all relevant dimensions.
• Lack of Complexity: Existing benchmarks often feature low scene density (few objects) and simple one-to-one reasoning (e.g., one action leads to one outcome). They fail to reflect real-world complexities involving high compositional density (many interacting objects) and multi-step reasoning chains (e.g., one action leading to multiple outcomes, or counterfactual scenarios).

Consequently, there is no standardized framework to holistically assess whether T2I models can not only "set the stage" (compose complex scenes) but also "direct the play" (reason through implicit causal and logical relationships).

2. Methodology: T2I-CoReBench

The authors introduce T2I-CoReBench, a comprehensive benchmark designed to evaluate T2I models under real-world complexities. The methodology involves three core components:

A. Evaluation Taxonomy (12 Dimensions)

The benchmark unifies composition and reasoning into a 12-dimensional taxonomy based on scene graphs and philosophical inference frameworks:

1. Composition (4 Dimensions):
   • Multi-Instance (MI): Generating ~25 distinct instances in a single scene.
   • Multi-Attribute (MA): Binding ~20 attributes to a single subject.
   • Multi-Relation (MR): Connecting ~15 spatial, comparative, or numerical relations.
   • Text Rendering (TR): Rendering ~15 texts with precise content and layout.
2. Reasoning (8 Dimensions):
   • Deductive: Logical Reasoning (LR), Behavioral Reasoning (BR), Hypothetical Reasoning (HR), Procedural Reasoning (PR).
   • Inductive: Generalization Reasoning (GR), Analogical Reasoning (AR).
   • Abductive: Commonsense Reasoning (CR), Reconstructive Reasoning (RR).

B. Data Construction Pipeline

• Prompt Generation: Instead of manual writing or rigid templates, the authors use Large Reasoning Models (LRMs) (e.g., Claude Sonnet 4, Gemini 2.5 Pro, OpenAI o3) to generate complex prompts. These prompts are designed with high semantic density and specific constraints (e.g., existential negation, narrative descriptions).
• Checklist Design: To enable fine-grained evaluation, every prompt is paired with a checklist of independent Yes/No questions (approx. 13,500 total questions). These questions decompose the scene into atomic, verifiable elements (e.g., "Is the object red?" rather than "Is the object correct?").
• Human Verification: All generated prompts and checklists undergo rigorous multi-round human verification to ensure logical consistency, visual feasibility, and alignment with the specific dimension guidelines.

C. Evaluation Protocol

• Models Evaluated: 38 state-of-the-art T2I models, including Diffusion (e.g., FLUX, SD), Autoregressive (e.g., Infinity), and Unified models (e.g., Janus, Show-o), covering both open-source and closed-source (e.g., GPT-Image, Seedream, Nano Banana).
• Scoring Mechanism: An MLLM evaluator (Gemini 2.5 Flash) is used to answer the checklist questions based solely on the generated image. The score is the ratio of correctly identified elements. This approach minimizes hallucination by treating evaluation as a binary visual question-answering task.

3. Key Contributions

1. First Holistic Benchmark: T2I-CoReBench is the first benchmark to systematically evaluate both composition and reasoning across 12 distinct dimensions with high complexity.
2. Granular Evaluation Framework: The introduction of human-verified checklists allows for precise error attribution, distinguishing between missing objects, wrong attributes, or failed reasoning steps.
3. Comprehensive Empirical Analysis: The paper provides the largest-scale evaluation to date (38 models, 1,080 prompts) and introduces a prompt-rewriting analysis to isolate the impact of native reasoning capabilities.

4. Key Results

The experiments reveal a stark disparity between composition and reasoning capabilities:

• Composition is Improving but Stalled:
  • Models show steady progress in composition. Top closed-source models (e.g., Nano Banana Pro) and open-source models (e.g., FLUX.2-dev) achieve high scores (~85-90%) on simple composition.
  • However, performance drops significantly in complex scenarios (high density). Even the best models struggle with Multi-Attribute binding (MA) and Multi-Relation generation (MR), indicating that fine-grained compositional control remains an open problem.
• Reasoning is the Critical Bottleneck:
  • Reasoning capabilities lag significantly behind composition. The best model (Nano Banana Pro) scores 82.7 on composition but only 75.3 on reasoning.
  • Open-source models show an even wider gap (e.g., Qwen-Image-2512: 83.7 composition vs. 51.7 reasoning).
  • Models struggle particularly with multi-step inferences (e.g., Behavioral Reasoning, Reconstructive Reasoning) and counterfactuals (Hypothetical Reasoning). They often fail to infer implicit outcomes (e.g., a squeezed tomato bursting) or apply rules to specific objects while ignoring others.
• Impact of Prompt Rewriting:
  • Rewriting prompts to make reasoning explicit (using an LLM to "think" before generating) improves performance for weaker models (gains of 20%+).
  • However, for strong models, the gains are marginal or negative, suggesting that textual reasoning alone is insufficient. The models fail because they cannot decouple the textual instruction from their inherent visual priors (e.g., knowing wheels are circular makes it hard to draw square wheels even when explicitly told).
• Architecture Trends:
  • Unified models (which integrate text and image generation) show intrinsic advantages in reasoning compared to pure diffusion models.
  • Strong instruction encoders (e.g., in Qwen-Image) correlate with better overall performance.

5. Significance and Future Directions

• Redefining T2I Evaluation: The paper shifts the focus from simple image quality to faithfulness to complex semantics, highlighting that current models are "painters" (good at composition) but poor "directors" (bad at reasoning).
• Identifying the Bottleneck: The study conclusively identifies reasoning as the primary barrier to the next generation of T2I models.
• Research Directions: The authors propose four future directions:
  1. Developing diverse, reasoning-oriented training data.
  2. Integrating LLM/MLLM reasoning capabilities directly into T2I pipelines.
  3. Adopting Chain-of-Thought (CoT) and self-consistency mechanisms during generation.
  4. Exploring multimodal interaction mechanisms (interleaving reasoning and generation) rather than just text-based prompting.

In summary, T2I-CoReBench provides a rigorous, high-fidelity benchmark that exposes the current limitations of AI in "directing the play," offering a clear roadmap for advancing T2I models from simple generators to intelligent, reasoning agents.