🤖 AI

Anchor-Conditioned Compositional Control for Landscape Image Generation

This paper introduces an anchor-conditioned finetuning framework for landscape image generation that utilizes a four-dimensional compositional anchor vector and decoupled cross-attention mechanisms to achieve superior horizon detection and rule-of-thirds alignment, demonstrating that compositional control precision is significantly enhanced by training on category-specific homogeneous scene subsets.

Original authors: Gadha Lekshmi P, Govind Arun, Rohith Syam, Ahmed Elgammal

Published 2026-06-09

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Gadha Lekshmi P, Govind Arun, Rohith Syam, Ahmed Elgammal

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a magical camera that can paint any landscape you can imagine just by typing a description. You might tell it, "Paint a mountain at sunset." But here's the catch: this camera is a bit of a wild artist. It decides where to put the horizon, how to split the sky from the ground, and where your eye should look. You can't really tell it, "Put the horizon low so the sky looks huge," or "Center the mountain perfectly." It just guesses based on what it has seen before.

This paper introduces a way to give that camera a "steering wheel" for composition. The authors call this an Anchor-Conditioned Framework.

Here is how it works, broken down into simple concepts:

1. The "Anchor" (The GPS Coordinates)

Before the camera starts painting, the researchers teach it to look at thousands of real photos and measure four specific things about how they are composed:

Where the horizon is: Is the sky taking up the top half, or just a sliver at the top?
How sure the computer is: Did it clearly see a horizon line, or was it fuzzy?
What's most important: Which part of the image is the "star" (the most eye-catching part)?
How much ground is visible: Is there a lot of foreground (rocks, trees) or is it mostly sky?

They turn these four measurements into a tiny, four-number code called an "Anchor Vector." Think of this like a set of GPS coordinates that tells the camera exactly where to place the horizon and how to frame the shot.

2. The "Decoupled" Pathway (The VIP Lane)

Usually, when you give a computer instructions, you just type a sentence (like "sunset mountains"). The computer reads this sentence and tries to guess the layout.

The problem is that if you just add a tiny note saying "put the horizon here" to the end of that sentence, the computer ignores it. It's like whispering a secret to a loud crowd; the main conversation (the text) drowns out the whisper.

The authors solved this by building a separate, VIP lane just for the Anchor.

The Old Way: Trying to squeeze the horizon instruction into the same text message as the description. (Result: The computer ignores it).
The New Way: Giving the Anchor its own dedicated highway (a "decoupled cross-attention" mechanism). The computer listens to the text description and the Anchor GPS coordinates at the same time, but the Anchor gets its own special attention so it can't be drowned out.

3. The "Fourier" Translator (Speaking the Computer's Language)

Computers are bad at understanding simple numbers like "0.3" (meaning the horizon is 30% down the screen). They prefer patterns.
The researchers use a trick called Fourier Encoding. Imagine taking that simple number and turning it into a complex, rhythmic song (using sine and cosine waves). This helps the computer "hear" the difference between a horizon at 30% and one at 31% much more clearly, allowing for very precise placement.

4. The Results: Does it Work?

The team tested this new camera against three other versions:

The Standard Camera: No special instructions.
The "Just Learn" Camera: A version that just practiced on landscape photos but didn't get the Anchor GPS.
The "Whisper" Camera: A version that tried to add the Anchor to the text sentence (the VIP lane was closed).
The New "Anchor" Camera: The one with the VIP lane and Fourier translator.

The Winner: The new Anchor Camera was the clear champion.

It placed the horizon exactly where the researchers asked for it (about 85% of the time).
It followed the "Rule of Thirds" (a famous photography rule for beautiful images) much better than the others.
The "Whisper" camera performed just as poorly as the standard camera, proving that the VIP lane is absolutely necessary.

5. The "Specialist" Discovery

The researchers also found something interesting: If you train the camera only on one type of landscape (like just deserts, or just forests), it gets even better at composing those specific scenes.

It's like hiring a chef who only cooks Italian food; they will make a better pasta dish than a chef who tries to cook everything from sushi to pizza.
When the camera was trained only on forests, it reduced errors by 40% compared to when it was trained on a mix of all landscapes.

Summary

This paper shows that you can teach an AI to be a better landscape photographer not by forcing it to memorize rules, but by giving it a dedicated control panel (the Anchor) that speaks a language the AI understands (Fourier encoding). This allows the AI to follow your compositional instructions precisely, rather than just guessing. It works best when the AI specializes in one specific type of scenery.

Technical Summary: Anchor-Conditioned Compositional Control for Landscape Image Generation

Problem Statement

While image generative models serve as powerful creative tools, they currently lack precise controllability over composition, a fundamental aspect of the creative process for photographers and visual artists. Unlike content specification, compositional intent involves decisions regarding horizon placement, frame division, and viewer focus. Standard diffusion models (e.g., Ho et al. 2020; Rombach et al. 2022) rely on statistical patterns absorbed during pretraining, offering no explicit mechanism for users to dictate spatial outcomes or compositional layout. Existing spatial control methods, such as ControlNet (Zhang et al. 2023) or GLIGEN (Li et al. 2023), often require structural inputs like depth maps, edge maps, or bounding boxes, which are not always available or desirable for high-level compositional direction.

Methodology

1. Compositional Anchor Vector Extraction

The core innovation is the automatic extraction of a four-dimensional anchor vector from training images without manual annotation. This vector encodes:

Horizon Position ( $y$ ): Estimated via Hough line transform on a Canny edge map of the upper 60% of the image, normalized to $[0, 1]$ .
Horizon Confidence: The fractional coverage of the longest detected horizontal line relative to image width.
Average Saliency: Computed via spectral residual analysis across top salient regions.
Foreground Ratio: A coarse estimate of visually prominent foreground content.

If no horizon is detected, defaults are applied ( $y=0.5$ , confidence $=0.0$ ) to preserve the informativeness of the remaining components.

2. Injection Architecture: Decoupled Cross-Attention

The anchor vector is injected into a fine-tuned latent diffusion model via a dedicated parallel attention pathway, distinct from the text conditioning stream.

Fourier Encoding: The 4D vector is passed through a Fourier embedding layer using 16 log-spaced frequencies ( $2^0$ to $2^{15}$ ) to create a 132-dimensional representation, enabling the learning of fine-grained spatial relationships from scalar inputs (motivated by GLIGEN ablation results).
Anchor Projection: A lightweight network (Linear $\to$ GELU $\to$ LayerNorm $\to$ Linear $\to$ LayerNorm) projects the 132D vector into a single anchor token of shape $[B, 1, 768]$ .
Decoupled Attention: At each of the 16 cross-attention layers in the UNet, separate Key and Value projection matrices (low-rank decomposition, rank 8) are introduced for the anchor. The output combines text and anchor attention:
$out = Attn(Q, K_{text}, V_{text}) + \lambda \cdot Attn(Q, K_{anchor}, V_{anchor})$
This design prevents the anchor token (representing $\approx 1.3\%$ of the attention budget) from being drowned out by the 77 text tokens, a failure mode observed when anchors are simply concatenated to the text sequence.

3. Training Strategy

Dataset: The Cropped-1901 Landscape Dataset (4,713 images) standardized to 1949 $\times$ 1024 and resized to 768 $\times$ 400 for training.
Three-Way Classifier-Free Guidance (CFG) Dropout: During training, a specific dropout scheme is applied: 5% drop both text and anchor, 5% drop only text, 5% drop only anchor, and 85% retain both. This enables conditional generation where the anchor can be specified at inference without manual spatial masks.
Fine-tuning: The model uses LoRA adapters (rank 16) on attention projections and is trained for 10,000 steps.

Key Results

Quantitative Performance

Evaluation was conducted across four model configurations: Baseline (no fine-tuning), LoRA-only (no anchor), Concatenation (anchor appended to text), and the Proposed (decoupled cross-attention).

Horizon Detection Rate: The proposed model achieved the highest rate (0.850), compared to 0.766 for the baseline and LoRA-only, and 0.766 for the concatenation variant.
Rule-of-Thirds Alignment: The proposed model achieved 0.817, significantly outperforming the baseline (0.500) and LoRA-only (0.636).
Horizon Deviation: The proposed model reduced compositional error to 0.062, a 15% reduction relative to the baseline (0.073).
Ablation on Concatenation: The concatenation variant performed identically to the unmodified baseline across all metrics, confirming that a single appended token is insufficient to influence spatial layout against 77 text tokens.

Category-Specific Ablation

The authors investigated whether training on compositionally homogeneous subsets improves precision. Three category-specific models (Mountain, Forest, Desert) were trained on subsets of the data.

Precision Gains: All category-specific models achieved 100% horizon detection and significantly lower deviation than the mixed-training model.
Forest Model: Achieved the lowest deviation (0.029), representing a 40% improvement over the mixed training model and a 60% improvement over the baseline.
Conclusion: Training on specific scene subsets reliably tightens conditioning precision, suggesting that compositional control is category-dependent.

Significance and Claims

The paper presents early results establishing that:

Decoupled Attention is Necessary: Effective compositional control via scalar descriptors requires a dedicated attention pathway with independent Key/Value projections; simple token concatenation fails.
Category Dependence: Compositional control precision is not uniform across all scenes; training on homogeneous subsets yields superior results compared to mixed training, even with fewer total steps.
Feasibility of Scalar Conditioning: It is possible to specify and measure compositional intent (horizon position, saliency, foreground ratio) at inference time without structural masks, using a compact 4D vector.

The authors acknowledge limitations, noting that the anchor vector is tailored to landscape imagery (governed by horizon/sky relationships) and may require redefinition for portraits or architecture. They also note that current evaluation relies on automatic metrics (horizon detection, rule-of-thirds) rather than human subjective judgment, which is planned for future user studies. The work does not claim to solve all compositional problems but demonstrates a viable framework for landscape-specific compositional control.