Agent Banana: High-Fidelity Image Editing with Agentic Thinking and Tooling

The paper introduces Agent Banana, a hierarchical agentic framework featuring Context Folding and Image Layer Decomposition to enable high-fidelity, multi-turn image editing at native 4K resolution, validated by the new HDD-Bench benchmark.

The Gist

Imagine you are a professional photo editor working on a massive, ultra-high-definition 4K poster for a movie. You need to make tiny, precise changes: "Change the blue bottle to red," then "Add a cat sitting on the table," then "Make the sky sunset-colored but keep the cat exactly where it is."

If you use a standard AI photo editor today, it's like hiring a very enthusiastic but clumsy intern. They might change the bottle to red, but in the process, they accidentally repaint the cat, blur the background, or shrink the whole image so the details look fuzzy. If you ask for a second change, they might forget what you did in the first step, or make the whole picture look "off" because they re-painted the entire image from scratch every time.

Agent Banana is the solution to this problem. It's a new AI system designed to act like a master craftsman with a perfect memory and a set of surgical tools, rather than a clumsy intern.

Here is how it works, broken down into simple concepts:

1. The Problem: The "Clumsy Intern" vs. The "Master"

Current AI editors suffer from three main issues:

• Over-editing: They change things you didn't ask for (like changing the cat's fur color when you only wanted to change the bottle).
• Memory Loss: They forget the history of your conversation. If you say "undo that," they might get confused about what "that" was.
• Resolution Drop: To save time, they often shrink your 4K masterpiece down to a small thumbnail, edit it, and blow it back up. This ruins the fine details (like the texture of wood or fabric).

2. The Solution: Agent Banana's "Brain" and "Hands"

Agent Banana splits the work into two distinct roles, like a Project Manager and a Specialized Technician.

• The Planner (The Project Manager): This part of the AI reads your vague request ("Make the scene look more like a movie") and breaks it down into a logical checklist. It doesn't touch the image; it just makes the plan.
• The Executor (The Technician): This part actually does the work. It picks the right tools and applies them.

3. The Secret Sauce: Two Magic Tricks

Trick A: "Context Folding" (The Perfect Memory)

Imagine you are writing a story. If you keep writing the whole story from page 1 every time you add a new sentence, you'll run out of paper (or computer memory) very quickly.

• How it works: Agent Banana doesn't re-read the whole history. Instead, it "folds" the past into a neat, structured summary. It remembers, "Okay, we changed the bottle to red, and the cat is now on the table," without needing to re-process the entire image history every time. This allows it to handle long, complex editing sessions without getting confused or forgetting earlier steps.

Trick B: "Image Layer Decomposition" (The Surgical Scalpel)

This is the most important part for high-quality editing.

• The Old Way: Imagine you want to change the color of a shirt on a person. The old AI takes the entire photo, blurs it slightly, paints the shirt, and puts the whole photo back together. The background gets slightly blurry every time you do this.
• The Agent Banana Way: It uses a "surgical scalpel." It identifies only the shirt, cuts it out onto a separate transparent sheet (a "layer"), changes the color on that sheet, and then glues it back perfectly. The background, the person's face, and the table remain untouched and crisp.
• The Result: You can edit a 4K image at its full, native resolution without ever losing a single pixel of detail.

4. The New Test: HDD-Bench

To prove this works, the creators built a new test called HDD-Bench.

• Think of previous tests as asking a student to solve a single math problem.
• HDD-Bench is like giving a student a complex, multi-step engineering project where they have to build a bridge, then add a road, then paint it, all while keeping the foundation intact. It tests if the AI can handle long chains of commands without messing up the parts it wasn't supposed to touch.

Why This Matters

In the real world, professional photographers and designers don't just want "good enough" edits. They need precision.

• If you are editing a movie scene, you can't have the background drift or blur after 10 different edits.
• If you are fixing a product photo, you can't have the texture of the product disappear.

Agent Banana bridges the gap between "fun AI toys" and "professional tools." It allows you to have a conversation with an AI, make complex changes step-by-step, and get back a result that looks like it was edited by a human expert with a high-end camera and Photoshop, but done instantly.

In short: Agent Banana is the AI editor that finally understands that sometimes you just want to change the color of a bottle without accidentally turning the whole world into a painting.

Technical Summary

1. Problem Statement

The paper identifies a significant gap between current generative image editing models and the requirements of professional workflows (e.g., photography, VFX, graphic design). Three persistent challenges are highlighted:

• Over-editing and Drift: Existing models often modify content beyond user intent, altering non-target regions (backgrounds, fixed objects) and causing "semantic drift" or texture degradation over multiple turns.
• Single-Turn Limitations: Most models are designed for single-turn interactions. Multi-turn editing often fails to maintain object faithfulness or handle complex, sequential dependencies where later edits rely on the state of previous ones.
• Resolution Mismatch: Current benchmarks and models typically operate at low resolutions (~1K), whereas professional workflows require native high-definition (4K/8K) editing. Downsampling and resampling lead to the loss of fine textures and sharp boundaries.

2. Methodology: The Agent Banana Framework

Agent Banana is a hierarchical agentic planner-executor framework designed for high-fidelity, multi-turn, native-resolution editing. It couples high-level reasoning with localized tool execution.

Core Architecture

The system operates in a multi-turn loop involving two specialized agents:

1. Planner: Interprets global user intent, decomposes complex "vibe" prompts into discrete, atomic sub-goals, and monitors overall progress.
2. Executor: Executes atomic editing operations using a "Photoshop-style" layer isolation approach. It includes a self-correction mechanism (Quality Test) to retry or rollback if an edit fails verification before presenting results to the user.

Key Mechanisms

To address context overflow and iterative degradation, the framework introduces two novel mechanisms:

• Context Folding (Memory Management):

  • Problem: Long interaction histories exceed LLM context limits and introduce noise.
  • Solution: A hierarchical memory abstraction that compresses history into three schemas:
    • Asset Level (ImageContext): A lightweight semantic node (URI, text description, parent ID) representing the image state, replacing raw image tokens.
    • Execution Level (ToolContext): Transient working memory for a single step (tool params, reasoning), which is discarded after completion to prevent noise.
    • Planning Level (ActionContext): Persistent memory retaining only verified effective editing paths and high-level semantics.
  • Result: Enables stable long-horizon planning without token explosion.

• Image Layer Decomposition (ILD) (Execution Strategy):

  • Problem: Full-image resampling causes pixel drift in unedited regions.
  • Solution: A "decompose-edit-fuse" paradigm.
    1. Decompose: Uses dynamic object-aware masks to crop the target region from the native high-resolution image into an isolated layer patch.
    2. Edit: Performs generation/inpainting only within the local coordinate system of the patch.
    3. Fuse: Blends the updated patch back into the original image using Gaussian blending.
  • Result: Preserves pixel-level fidelity in non-target regions and supports ultra-HD editing beyond model native limits.

• Atomic Action Space: The Executor supports five primitive operations: replace, remove, add, adjust (attributes), and undo (rollback via state graph).

3. Key Contributions: HDD-Bench

To rigorously evaluate these capabilities, the authors introduce HDD-Bench (High-Definition, Dialogue-based Benchmark).

• Features:
  • Multi-Turn & Logically Dependent: Simulates professional workflows with 3-turn instruction chains where each turn induces a verifiable state transition.
  • Native 4K Resolution: Contains images at 11.8M pixels (approx. 4K), bridging the gap between research and industry standards.
  • Verifiable Ground Truth: Uses a scalable symbolic data engine to generate deterministic state transitions ($s_{t+1} = T(s_t, c_t)$). This allows for precise, step-by-step verification of object attributes without expensive human pixel-level annotation.
• Evaluation Protocol:
  • State-based Metrics: Instruction Following (IF) and Image Consistency (IC) based on object attribute correctness.
  • Visual Fidelity: Uses Otsu-Masked Metrics (PSNROM, SSIMOM, LPIPSOM) to isolate and measure the preservation of non-edited background regions, avoiding the pitfalls of global metrics that penalize valid foreground changes.

4. Experimental Results

The paper evaluates Agent Banana against state-of-the-art baselines (e.g., Flux.1 Kontext, GPT-Image-1, Nano Banana Pro, Qwen-Image) on HDD-Bench and ImgEdit-Bench.

• Multi-Turn Consistency: Agent Banana achieves the best performance in Image Consistency (IC: 0.871) and Instruction Following, demonstrating superior ability to maintain non-target regions across turns.
• High-Fidelity Preservation: It achieves the highest SSIMOM (0.84) and lowest LPIPSOM (0.12), indicating minimal degradation of background textures compared to baselines.
• Native Resolution: Agent Banana is one of the few models capable of processing native 4K images without downsampling, preserving high-frequency details that baselines lose.
• Single-Turn Performance: Despite its complex agentic architecture, it remains competitive on standard single-turn benchmarks (ImgEdit-Bench), proving the framework does not sacrifice atomic precision.
• Ablation Studies:
  • LLM Capability: Replacing the base LLM with a smaller model (Qwen-3-8B) caused significant failure in planning and parameter generation, highlighting the need for strong reasoning capabilities.
  • PIED (Prior-Induced Editing Drift): The study identifies a failure mode where repeated re-synthesis causes background drift. Agent Banana effectively mitigates this, keeping metrics stable across turns, whereas baselines show accumulating artifacts.

5. Significance and Impact

• Professional Viability: Agent Banana moves image editing from "toy demos" to professional-grade tools by solving the resolution and multi-turn reliability issues inherent in current generative models.
• New Evaluation Paradigm: HDD-Bench establishes a new standard for evaluating long-horizon, high-fidelity editing, shifting focus from single-turn visual appeal to state tracking, edit locality, and drift prevention.
• Agentic Workflow: The paper demonstrates that decomposing editing into a planning-execution loop with self-reflection is essential for complex, sequential tasks.
• Ethical Considerations: The authors emphasize the need for provenance and consent mechanisms, noting that while these tools improve controllability, they also carry risks for non-consensual manipulation. The framework is designed to measure controllability and detect failure accumulation rather than optimize for unconstrained manipulation.

In summary, Agent Banana represents a shift toward reliable, state-aware, and high-resolution image editing by leveraging agentic reasoning, hierarchical memory, and localized layer processing, validated by a rigorous, professional-grade benchmark.