🤖 AI

Visual Accommodation: Rethinking Image Scale as a Learnable Variable for Object Detection

The paper introduces Ciliary-DETR, a novel object detection framework that mimics biological visual accommodation by employing a lightweight, learnable scale predictor to dynamically optimize inference resolution, thereby overcoming the limitations of fixed input sizes and enhancing robustness through loss-driven scale adaptation.

Original authors: Daeun Seo, Hoeseok Yang, Sihyeong Park, Hyungshin Kim

Published 2026-05-14

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Daeun Seo, Hoeseok Yang, Sihyeong Park, Hyungshin Kim

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to recognize objects in a photo, like finding a tiny ant on a leaf or a giant elephant in the distance. In the world of computer vision, most AI detectors are like a person wearing fixed prescription glasses. No matter what they are looking at, the "lens" (the image resolution) stays the same. If the object is too small, it's blurry; if it's too big, it's cut off. To fix this, current AI usually tries on many different pairs of glasses (checking the image at many different sizes) and picks the best one, but this is slow and computationally expensive.

This paper introduces Ciliary-DETR, a new way to teach AI to adjust its own "glasses" on the fly, just like your eyes do.

The Biological Analogy: The Eye's "Focus Muscle"

In your human eye, there is a tiny muscle called the ciliary muscle. When you look at something close, this muscle squeezes to change the shape of your lens, bringing the image into sharp focus. When you look far away, it relaxes. This process is called accommodation.

The authors propose that an AI detector should have a similar "ciliary muscle." Instead of being stuck with one fixed image size, the AI should have a lightweight "scale predictor" that acts like this muscle. It looks at the image and instantly decides: "This scene needs to be zoomed in," or "This scene needs to be zoomed out," before the main detection brain even starts working.

The Big Problem: "How do we teach it?"

Here is the tricky part: In a standard training class, the teacher (the computer) doesn't actually know the "perfect" zoom level for every single photo. It's like asking a student to guess the perfect focus setting without a reference answer. If you just ask the AI to guess, it might get confused.

To solve this, the authors invented a clever two-part training system:

The "Size" Coach (Scale Loss): Imagine the AI is learning that "small things need zooming in" and "big things need zooming out." The system creates a rule where the AI gets a "score" based on how well it balances these needs. It learns to treat the zoom level as a probability, gently nudging the image size to make small objects bigger and large objects smaller, without needing a specific "correct" answer for every photo.
The "Performance" Coach (Distribution Loss): This is the smartest part. The AI watches its own performance. It asks, "When I look at images of this size, do I make fewer mistakes?" It builds a mental map (a statistical distribution) of which sizes work best for the objects it sees. It then adjusts its "muscle" to aim for that sweet spot where it performs best.

How It Works in Practice

Think of the AI pipeline as a factory assembly line:

Old Way: The factory takes a photo, runs it through the machine at a fixed size, and if it misses something, it has to re-run the whole photo at a different size. This is slow.
Ciliary-DETR Way: Before the photo hits the main machine, a tiny, fast "pre-processor" (the scale predictor) looks at the photo and says, "This one needs to be 1.2x bigger." It resizes the photo instantly. Then, the main machine processes this perfectly sized image just once.

The Results: Faster and Smarter

The paper tested this on standard object detection datasets (like finding cars, people, and animals).

Efficiency: Because the AI only needs to process the image once (instead of trying many sizes), it saves a significant amount of computing power (about 17–19% less energy).
Accuracy: Surprisingly, it didn't just save energy; it actually got better at finding objects. By dynamically adjusting the zoom, it handled both tiny and huge objects better than the fixed-size models.
Flexibility: The system is so adaptable that it can be "plugged in" to existing AI models that were never trained with this feature, and they still work better.

Summary

In simple terms, Ciliary-DETR gives AI the ability to "squint" or "widen its eyes" depending on the scene, mimicking how human eyes naturally focus. It does this without needing a human to tell it exactly how to zoom, learning instead by observing what works best for the objects it sees. The result is an AI that sees the world more clearly while using less energy.

Technical Summary: Ciliary-DETR

Problem Statement
Modern object detection networks, particularly those based on the DETR (DEtection TRansformer) architecture, typically operate on a fixed input resolution during inference. While multi-scale data augmentation during training improves robustness to scale variations, the inference phase lacks sample-specific adaptability. Unlike the human visual system, which dynamically adjusts focal depth (accommodation) to maintain stable perception, current detectors treat image resolution as a static design choice rather than a dynamic, data-dependent variable. Existing adaptive scaling methods often rely on multi-branch architectures, multi-inference strategies, or predefined resolution sets, which hinder scalability and end-to-end optimization. Furthermore, the optimal input scale for a specific image is inherently unobservable and lacks explicit supervision, making direct scale optimization ill-posed under standard training setups.

Methodology: Ciliary-DETR
The authors propose Ciliary-DETR, a framework that treats image resolution as a learnable variable analogous to the biological ciliary muscle adjusting the eye lens. The method introduces a lightweight Scale Predictor module placed prior to the detection backbone to dynamically estimate an input-dependent scale factor ( $\phi$ ).

Architecture: The scale predictor consists of a lightweight backbone (ResNet-18) and a prediction head. It takes an input image, down-samples it to reduce computational overhead, and outputs a scalar scale factor constrained within a desired range $[\tau_{min}, \tau_{max}]$ . This factor is used to resize the image via bilinear interpolation before it is passed to the detection network (e.g., DN-DETR).
Training Strategy: The framework employs a two-stage training process leveraging a "supernet" philosophy:
1. Multi-Scale (MS) Training: The detection network is initially trained with MS data augmentation to build robustness against scale variations.
2. Joint Optimization: The scale predictor and the detection network are jointly trained. During this phase, the resolution range is varied (subnet training) to explore performance-efficiency trade-offs while sharing weights.
Optimization Objectives: To address the lack of ground-truth scaling labels, the authors introduce two complementary, loss-driven objectives:
1. Scale Loss: A probabilistic formulation based on Binary Cross-Entropy (BCE). It models the preference for up-scaling or down-scaling as a function of object size, using anchor sizes ( $A_{min}, A_{max}$ ) to define saturation regions. This loss encourages the network to up-scale small objects and down-scale large ones to normalize their effective sizes.
2. Distribution Loss: This loss aligns the scaling decisions with downstream detection performance without explicit supervision. It models the relationship between object scale and detection loss using a learnable Beta distribution. By minimizing the 1-Wasserstein distance between this parametric distribution and an empirical observation distribution (derived from detection losses), the network learns to adjust the anchor sizes ( $A_{min}, A_{max}$ ) to match the performance-sensitive regions of the data.

Key Contributions

Learnable Image Scale: The paper proposes a novel formulation where image scale is treated as a directly learnable variable, enabling dynamic, data-dependent resolution adjustment within a single forward pass.
Loss-Driven Optimization: It introduces a parametric, probabilistic framework for scaling behavior that does not require explicit scale annotations. The combination of Scale Loss and Distribution Loss enables stable, data-driven adaptation.
Single-Pass Multi-Scale Reasoning: The method bridges the gap between training-time robustness (via MS augmentation) and test-time adaptation, achieving effective multi-scale reasoning without the computational overhead of explicit multi-resolution inference.
Modularity and Compatibility: The framework is designed to be compatible with modern DETR-based detectors without requiring architectural modifications to the detection pipeline itself, preserving the simplicity of the detection network.

Experimental Results
Extensive experiments on the MS COCO and LVIS datasets demonstrate the effectiveness of Ciliary-DETR:

Performance-Efficiency Trade-off: Compared to strong baselines (e.g., DN-DETR-R50 and Swin-S), Ciliary-DETR achieves consistent accuracy gains (e.g., +1.3 AP on R50, +1.1 AP on Swin-S) while reducing computational complexity (GFLOPs) by approximately 17–19%.
Zero-Shot Generalization: The scale predictor can be applied zero-shot to unseen detection architectures (e.g., Deformable-DETR, DINO-DETR, H-DETR), reducing GFLOPs by 18–19% while maintaining or slightly improving AP.
Comparison with TTA: The method outperforms random resolution selection and naive multi-scale test-time augmentation (TTA). It achieves results close to a brute-force search (BFS) for optimal resolution (within a 0.4–0.5 AP gap) but does so in a fully learnable, single-pass manner.
Robustness: The model demonstrates strong robustness across a wide range of resolutions at test time, outperforming fixed-resolution baselines even at resolutions outside the training range.

Significance and Claims
The paper claims that Ciliary-DETR offers an orthogonal direction to traditional architectural optimization by focusing on input-level adaptation. By emulating biological visual accommodation, the framework enables flexible and efficient single-pass inference that dynamically adjusts to the content of each image. The authors emphasize that this approach bridges the gap between training-time augmentation and test-time adaptation, allowing for scalable deployment without hindering the modularity of modern detection pipelines. The work suggests that treating resolution as a learnable variable is a viable and effective strategy for improving both the accuracy and efficiency of object detection systems.