💻 computer science

PPSNet: Perceptual Prior Similarity-guided Network for Class-agnostic Counting

This paper introduces PPSNet, a novel class-agnostic counting framework that theoretically validates the Extract-and-Match paradigm and addresses the limitations of bounding box-based visual exemplars by employing a Perceptual Prior Construction Module to refine scale distribution and a Similarity-guided Weighting Module to enhance feature matching, thereby achieving superior performance across visual, text, and zero-shot prompting scenarios.

Original authors: Shengwei Jia, Junhui Liu, Jiahao Wang, Yongqiang Cui, Shihui Zhang

Published 2026-08-12

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Shengwei Jia, Junhui Liu, Jiahao Wang, Yongqiang Cui, Shihui Zhang

Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a detective trying to solve a mystery in a crowded room. Your job is to count how many people are wearing red hats. In the old days, if you wanted to count blue hats, you'd have to hire a whole new team of detectives who only knew how to spot blue. But what if you could teach one detective to count any hat, just by showing them a single example? This is the world of "class-agnostic counting," a branch of computer vision where machines learn to count objects of any kind—birds, cars, or cookies—without needing a specific training manual for each new type.

To do this, the detective usually needs a "prompt," a little hint to tell them what to look for. Sometimes this hint is a sentence like "count the seagulls," or sometimes it's just the machine guessing on its own. But the most accurate method so far has been "vision-guided" counting: showing the computer a few tiny pictures (exemplars) of the object you want to count, usually drawn inside a square box. Think of it like handing the detective a photo of a seagull and saying, "Find everyone who looks like this." The problem is, the square box is a bit clumsy. It often grabs a chunk of the ocean or the sky along with the bird. As the computer gets deeper into its thinking, it starts getting confused by that extra background noise, thinking the water is part of the bird, which leads to a bad count.

This paper introduces a new detective team called PPSNet (Perceptual Prior Similarity-guided Network) that fixes these clumsy mistakes. The researchers argue that the old way of working—where the computer first studies the photo and the example separately, and then tries to match them—is like trying to solve a puzzle by looking at the pieces in one hand and the picture in the other, without ever letting your hands touch. They propose a new way where the computer looks at the photo and the example together from the very start, allowing them to talk to each other and refine their understanding instantly.

But PPSNet doesn't just change how they look; it changes what they see. The team noticed that the square boxes used to grab the examples were too messy. So, they built a special tool called the Perceptual Prior Construction Module. Imagine if, instead of just handing the detective a blurry, stretched-out photo of a bird, you also gave them a mental map—a glowing "heat map"—that shows exactly where the bird is inside that messy box and where the ocean starts. This map tells the computer, "Ignore the water; focus on the bird." This stops the computer from getting distracted by the background noise that usually tricks it.

Furthermore, the team added a Similarity-guided Weighting Module. This acts like a spotlight. Once the computer finds a spot in the big photo that looks like the example, this module turns up the brightness on that spot and dims everything else. It's like the detective saying, "Aha! This looks exactly like the bird I was shown!" and ignoring the rest of the room. They also proved that this "spotlight" works best when they train the computer to pay attention to the difference between the bird and the background, using a technique similar to how humans learn to tell things apart by comparing them.

The results are impressive. When tested on massive datasets containing thousands of images of everything from parking lots to crowds of people, PPSNet made fewer mistakes than any previous method. For example, on a test set of 6,135 images, the new method reduced the average counting error significantly compared to the previous best. It even worked when the researchers removed the visual examples entirely, letting the computer count based on just a text description or no hint at all, though it still performed best when it could see a few examples.

The authors are careful to note that while their method is a major step forward, it's not magic; it still relies on the computer seeing the image and the example. They also point out that the old method of using square boxes to grab examples is inherently flawed because it forces irregular shapes into a rigid frame, losing important details. PPSNet doesn't just patch this hole; it builds a new foundation that understands the shape and distribution of the object better than the box ever could. By combining a "mental map" of the object's location with a "spotlight" that highlights matches, this new network helps computers count the world with a level of precision that was previously out of reach.

Technical Summary: PPSNet for Class-Agnostic Counting

Problem Statement
Class-agnostic counting (CAC) aims to estimate the number of objects of arbitrary classes using a single pre-trained model with minimal or no prompts. While vision-guided exemplar counting (using bounding boxes to select target examples) currently offers the highest accuracy, it suffers from two critical limitations inherent to the standard "Extract-then-Match" paradigm and the use of bounding boxes:

Background Noise Pseudo-activation: Bounding boxes inevitably include background regions due to geometric constraints. As network depth increases, this background noise is progressively activated during feature extraction, leading to erroneous matching between visual exemplars and the query image.
Scale Information Loss: To feed exemplars into fixed-size networks, current methods resize exemplars to a uniform dimension. This operation distorts the aspect ratios of elongated objects, causing a loss of intrinsic scale and structural information.
Lack of Bidirectional Interaction: The dominant "Extract-then-Match" paradigm extracts features from query images and exemplars independently before matching them. This neglects the bidirectional interaction between query and exemplar features during the extraction phase, resulting in query features that lack target awareness and are susceptible to background interference.

Methodology
The authors propose PPSNet (Perceptual Prior Similarity-guided Network), which adopts the Extract-and-Match paradigm. Unlike Extract-then-Match, this approach concatenates query and exemplar tokens at the input stage, allowing for joint feature extraction and similarity modeling via self-attention. This enables true bidirectional information flow, where query features are updated with exemplar information from the initial layers, enhancing target awareness.

PPSNet integrates two core modules to address the specific limitations of bounding box-based prompts:

Perceptual Prior Construction Module (PPCM):
- Goal: To characterize the distribution of visual exemplars within bounding boxes and mitigate the effects of resizing and background noise.
- Mechanism: PPCM constructs a Gaussian prior based on the original dimensions of the visual exemplars before resizing. It calculates the relative displacement of pixels from the center and modulates the prior strength using an area coefficient (inverse to the exemplar's size).
- Effect: This prior is concatenated with the exemplar images. It implicitly encodes the original aspect ratio information (compensating for geometric distortion caused by resizing) and emphasizes the central distribution of the target within the bounding box, thereby reducing the activation of background noise during feature extraction.
Similarity-guided Weighting Module (SWM):
- Goal: To explicitly supervise the similarity between query and exemplar features and enhance feature responses in target regions.
- Mechanism: SWM computes a cosine similarity map between the query features and visual exemplar features. It constructs a patch-level foreground mask based on the ground truth density map to supervise this similarity map using a contrastive learning loss ( $L_{sim}$ ).
- Effect: The resulting similarity map is used to weight the query features. This process amplifies the response of query features in regions that semantically match the visual exemplars while suppressing background responses.

The final objective function combines the standard counting loss ( $L_{count}$ ) and the similarity loss ( $L_{sim}$ ), balanced by a factor $\lambda$ .

Key Contributions

Theoretical Analysis: The paper provides a theoretical analysis demonstrating the advantages of the Extract-and-Match paradigm over Extract-then-Match. It demonstrates how joint encoding overcomes the lack of bidirectional information flow guidance in Extract-then-Match, endowing query features with target awareness from the start, which Extract-then-Match fails to achieve due to its sequential nature.
Addressing Background Noise: The authors identify and address the "background noise pseudo-activation" problem caused by the geometric constraints of bounding boxes. They propose the PPCM as the first work to explicitly model the distribution of exemplars within bounding boxes to mitigate this issue and compensate for scale information loss during resizing.
Novel Weighting Mechanism: The SWM introduces patch-level supervision for similarity maps using contrastive learning. This allows the network to explicitly learn to distinguish target regions from background and use this knowledge to weight query features, enhancing discriminability.
Generalization: The framework is extended to support guidance-less counting (using learnable tokens) and text-guided counting (using CLIP text embeddings), demonstrating versatility beyond vision-guided prompts.

Experimental Results
Extensive experiments were conducted on the FSC-147, CARPK, and COCO datasets.

FSC-147 (Vision-guided): PPSNet achieved state-of-the-art performance. In the single-exemplar setting, it reduced MAE to 11.12 and RMSE to 81.17 on the test set. In the three-exemplar setting, it achieved an MAE of 9.29 and RMSE of 61.83, outperforming the previous best (RCFNet) and the Extract-then-Match baseline (CounTR) by significant margins (e.g., 22.26% MAE improvement over CounTR).
Guidance-less & Text-guided: When adapted to guidance-less and text-guided paradigms, PPSNet also achieved state-of-the-art results, outperforming methods like GCA-SUN and T2ICount.
Cross-Dataset Generalization: On the CARPK dataset (without fine-tuning), PPSNet outperformed BMNet+ and CounTR. On Val-COCO and Test-COCO, it surpassed both regression-based counting methods and object detection baselines (Faster R-CNN, RetinaNet).
Ablation Studies: Removing either PPCM or SWM resulted in performance degradation, confirming the mutual reinforcement of both modules. The study also determined that 3 visual exemplars yield optimal performance, while 2 learnable tokens are optimal for guidance-less counting.

Significance and Claims
The paper claims that PPSNet represents a significant advancement in class-agnostic counting by theoretically analyzing the Extract-and-Match paradigm and practically solving the long-standing issue of background noise interference in bounding box-based prompts. The authors emphasize that this is the first work to address the non-negligible impact of bounding box geometric constraints on counting accuracy through perceptual priors and similarity-guided weighting. The results demonstrate that by enhancing target awareness and suppressing background noise, the model achieves superior accuracy and generalization across diverse scenarios and prompt types (vision, text, and none).

Technical Summary: PPSNet for Class-Agnostic Counting

More like this