💻 computer science

CoDi -- an exemplar-conditioned diffusion model for low-shot counting

The paper introduces CoDi, a novel exemplar-conditioned latent diffusion model that significantly outperforms existing state-of-the-art methods in low-shot object counting by effectively addressing challenges in dense, small-object regions through a specialized conditioning module that generates high-quality density maps for precise localization.

Original authors: Grega Šuštar, Jer Pelhan, Alan Lukežič, Matej Kristan

Published 2026-07-17

📖 7 min read🧠 Deep dive

CC BY 4.0

Original authors: Grega Šuštar, Jer Pelhan, Alan Lukežič, Matej Kristan

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to count every single grain of sand on a beach, or every tiny star in a crowded night sky. If you just look at the whole picture, it's a blur. If you try to count them one by one with a magnifying glass, you might miss some or count the same one twice. This is the daily struggle for computers trying to "count" things in images, a task known as object counting. For a long time, computers had two main ways to do this. The first was like pouring water over the sand: they created a "density map," a smooth hill where the height represented how many things were there. It was good for getting a rough total, but terrible at telling you exactly where each grain was. The second way was like using a laser pointer to tag every single star. This was great for finding locations, but if the stars were too crowded, the computer would get confused, run out of "pointers" to use, and start missing things or getting stuck in a loop.

Now, imagine a new kind of artist who doesn't just paint a blurry hill or use a limited number of laser pointers. Instead, this artist starts with a canvas full of static noise—like a TV screen with no signal—and slowly, step-by-step, clears away the noise to reveal a perfect, sharp picture of exactly where every object is. This is the core idea behind a new method called CoDi (Counting by Diffusion). The researchers behind this paper, Grega Šuštar and his team, have built a system that uses a "diffusion model"—a type of AI famous for generating images—to solve the counting problem. Instead of guessing a number or drawing a box, CoDi learns to "denoise" a blank canvas until it pops out with tiny, distinct dots, each representing one object. The magic trick is that it can do this even if you only show it one or two examples of what to look for, or even if you give it no examples at all. It's like showing a friend a single photo of a specific type of mushroom and asking them to find every single one of those mushrooms in a forest; CoDi doesn't just guess the total number, it points to the exact spot of every single mushroom, even in the thickest, most crowded parts of the forest.

The Problem: The "Crowded Room" Dilemma

Counting things in a photo is easy when the objects are spread out. But what happens when you have a photo of a swarm of bees, a pile of grapes, or a crowd of people? This is where old methods hit a wall.

The "density" methods are like trying to count a crowd by measuring the height of the crowd's "heat." If the crowd is dense, the heat is high, and the computer knows there are many people. But it can't tell you who is where. If you try to find individual people by looking for the highest peaks in that heat map, you often get it wrong because the heat blurs together.

The "detection" methods are like having a security guard with a limited number of walkie-talkies. The guard can point to specific people, but if there are too many people (more than the number of walkie-talkies), the guard has to split the crowd into smaller groups, count them separately, and try to stitch the answers back together. This is slow, messy, and often leads to counting the same person twice or missing someone entirely.

The Solution: CoDi's "Magic Noise"

The authors propose CoDi, a system that treats counting like a game of "cleaning up static."

Here is how it works, step-by-step:

The Starting Point: Imagine a blank, noisy TV screen. This is the starting point for CoDi. It doesn't know where the objects are yet.
The "Exemplar" Clue: If you want to count apples, you show the computer a few pictures of apples (called "exemplars"). CoDi has a special "conditioning module" that acts like a super-smart filter. It looks at your apple examples and says, "Okay, I need to find things that look exactly like these, and ignore everything else."
The Denoising Dance: CoDi starts removing the noise from the TV screen, but it does it in a very specific way. It doesn't just guess; it gradually refines the image. With every step, the blurry noise turns into sharper and sharper dots.
The Result: By the end of the process, the screen isn't a blurry hill or a messy list. It's a clean map with tiny, sharp peaks. Each peak is a perfect dot representing one object. The computer then just counts the dots.

The secret sauce is that CoDi learns to generate these sharp dots one by one rather than averaging them out. This means even in a super-crowded scene where objects are touching, CoDi can separate them and count them accurately.

What They Found: Beating the Best

The team tested CoDi on some very difficult datasets, including FSC147 (which has 147 different types of objects) and MCAC (a multi-class counting dataset). The results were impressive:

Few-Shot Counting: When given just a few examples (like 3 images of the target object), CoDi beat the current best methods by 15% in terms of accuracy (Mean Absolute Error, or MAE).
One-Shot Counting: Even when given only one example, CoDi was still the best, beating the previous top method by 13%.
No Examples (Reference-less): When given no examples at all and asked to count the most common object in the image, CoDi still outperformed the best existing methods by 10%.
The Multi-Class Challenge: On the MCAC dataset, where an image might have apples, oranges, and bananas all mixed together, CoDi crushed the competition, beating the top method by a massive 38%.

Why This Matters

The paper explicitly rules out the idea that you need a massive number of pre-trained "pointers" (like the old detection methods) to count huge crowds. They showed that tiling images (cutting them into pieces) is a clunky workaround that slows things down and causes errors. CoDi proves you don't need to cut the image up; you can process the whole thing at once and still get perfect results.

They also showed that CoDi is robust. Even if the examples you give it are slightly messy or the objects are tiny, it holds up. In fact, the authors found that CoDi is so stable that if you run the same image through it twenty times, the count changes by less than 1% on average.

The Limits and the Future

While CoDi is a huge step forward, the authors are honest about where it struggles. It sometimes has trouble with objects that are extremely long and thin (like a row of books standing on their edge) or when objects are overlapping in very strange ways. They also note that while it's great at finding the center of an object, it doesn't yet draw a box around the whole object (though they plan to work on that).

In short, CoDi suggests that by treating counting as a process of "cleaning up noise" rather than "guessing a number," we can build computers that see crowds the way humans do: not as a blur, but as a collection of distinct, countable individuals. It's a new way of seeing the world, one dot at a time.

Technical Summary: CoDi – An Exemplar-Conditioned Diffusion Model for Low-Shot Counting

1. Problem Definition

The paper addresses low-shot object counting, a task that involves estimating the number of objects from unseen categories in an image using only a few annotated exemplars (few-shot/one-shot) or no exemplars at all (reference-less).

While modern methods have progressed, they face significant challenges in dense regions containing small objects:

Density-based counters (e.g., LOCA, CounTR) estimate counts by summing a regressed density map. However, these maps often lack well-expressed single modes for individual objects, leading to poor localization and false positives when Non-Maxima Suppression (NMS) is applied.
Detection-based counters (e.g., CountGD, GeCo) predict bounding boxes or center points. These methods are often query-based (transformer architectures) with a fixed number of pre-trained queries. In dense scenes, the object count often exceeds this capacity, forcing the use of computationally expensive tiling (splitting images into crops) and complex boundary-reconciliation post-processing. Furthermore, regression-based models trained on noisy ground-truth centers often produce blurred maps unsuitable for accurate peak counting.

2. Methodology: CoDi

The authors propose CoDi (Counting by Diffusion), the first latent diffusion-based low-shot counter designed to produce high-quality, crisp object location maps.

Core Architecture

CoDi utilizes a Conditioned Latent Diffusion Model (LDM) to iteratively refine an object location map.

Input Representation:
- The input image $I$ is encoded into a latent representation $z_I$ using a pre-trained AM-RADIO backbone.
- The target object locations map $L$ is constructed by placing normalized Gaussians over ground-truth centers. This map is projected into a latent space $z_0$ via a pre-trained VAE.
- During the forward diffusion process, noise is added to $z_0$ to create a sequence of noisy latents $z_t$ .
Denoising Network:
- A standard UNet (three levels) is trained to reverse the diffusion process, predicting the noise or the velocity vector ( $v_t$ ) to recover the clean latent map.
- The input to the UNet is the concatenation of the image latent $z_I$ and the current noisy location latent $z_t$ .
Exemplar-Based Conditioning (EC):
- This is the core innovation. To specialize the model to the target object category at test time, the network is conditioned on user-provided exemplar bounding boxes.
- Adaptive Prototype Extraction: Unlike static conditioning, CoDi extracts appearance prototypes (via RoI-Align on current UNet features) and shape prototypes (via MLP on box dimensions) at each diffusion timestep.
- Modulation: These prototypes are injected into the first and last layers of the UNet via a Transformer attention layer. The UNet features act as queries, while the time-specific prototypes act as keys and values. This allows the network to continuously adapt its understanding of the target object as the denoising process refines the location map.
Inference:
- Starting from random Gaussian noise, the network performs $T$ denoising steps (e.g., $T=25$ ).
- The final latent map is decoded by the VAE into a high-resolution location map $\tilde{L}$ .
- Object centers are detected as local maxima in $3\times3$ neighborhoods exceeding a threshold, and the total count is the number of detected peaks.

Reference-Less Adaptation

For scenarios without exemplars, the authors introduce trainable reference-less queries. These queries are converted into prototypes by attending to the layer features, allowing the model to count the majority class without specific visual prompts. This requires only fine-tuning the query parameters while freezing the rest of the network.

3. Key Contributions

First Latent Diffusion Low-Shot Counter: CoDi is the first method to apply latent diffusion specifically for low-shot counting, avoiding the annotation noise averaging problem of regression-based models by generating single-instance predictions.
Exemplar-Based Conditioning Module: A novel mechanism that extracts and adjusts object prototypes dynamically during the diffusion process. This leads to narrow, separated peaks in the location map, enabling accurate localization via NMS without the need for tiling.
Unified Low-Shot Spectrum: CoDi is the first diffusion-based counter to cover few-shot, one-shot, and reference-less scenarios within a single framework.
Efficiency: By operating in latent space and avoiding the adaptive tiling required by query-based detectors (like CountGD), CoDi offers a more computationally stable inference process, particularly for images with many small objects.

4. Experimental Results

The paper evaluates CoDi on the FSC147 (few-shot) and MCAC (multi-class) benchmarks.

Few-Shot (FSC147): CoDi outperforms the state-of-the-art (SOTA) density-based counter (CountDiff) by 38% MAE and 42% RMSE on the test set. Compared to the localization-based SOTA (CountGD), CoDi improves MAE by 15% and RMSE by 29%.
One-Shot: CoDi achieves a new SOTA, outperforming GeCo by 13% MAE and 49% RMSE, demonstrating robustness even with a single exemplar.
Reference-Less: CoDi outperforms the best text-free SOTA (GeCo) by 10% MAE. While it trails behind text-prompted zero-shot methods (e.g., CountSE), it remains competitive without relying on text.
Multi-Class (MCAC): CoDi sets a new SOTA, outperforming the top method (GeCo2) by 38% MAE.
Localization Quality: CoDi achieves higher recall (0.88 vs 0.81) and F1 scores (0.92 vs 0.87) compared to CountGD on the test set, with an average detection distance of 8.79 pixels from ground truth.
Efficiency: CoDi shows stable inference times (~1.1s/image) with low variance, unlike CountGD whose runtime degrades significantly on dense images due to tiling. It uses fewer parameters than GeCo and less VRAM.

5. Significance and Claims

The authors claim that CoDi represents a significant shift in low-shot counting by reformulating the task as a generation problem rather than a regression or detection problem.

Robustness to Density: The primary significance lies in CoDi's ability to handle dense regions of small objects where existing methods fail due to query limits or blurred density maps. The diffusion process naturally produces "crisp" peaks suitable for NMS.
Elimination of Tiling: By avoiding the fixed-query capacity limit of transformer detectors, CoDi removes the need for computationally heavy tiling and boundary reconciliation.
Adaptability: The exemplar conditioning module allows the model to generalize to unseen categories with minimal data, while the reference-less adaptation extends this to majority-class counting without retraining the entire network.

The paper concludes that CoDi sets a new state-of-the-art across the low-shot counting spectrum, offering a balance of high accuracy, precise localization, and computational stability. Future work is noted to potentially extend this to bounding box prediction and handling objects with extreme aspect ratios.