💻 computer science

Falcon Perception-HD: High Density Perception via Reinforcement Learning

This paper introduces Falcon Perception-HD, a reinforcement learning framework that aligns autoregressive perception models with evaluation metrics to achieve state-of-the-art performance in extremely dense scenes, eliminate common issues like mask repetition and the need for NMS, and robustly detect object existence without negative training samples.

Original authors: Sofian Chaybouti, Yasser Dahou, Ngoc Dung Huynh, Reda Alami, Hilde Kuehne

Published 2026-08-20

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Sofian Chaybouti, Yasser Dahou, Ngoc Dung Huynh, Reda Alami, Hilde Kuehne

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

In the world of computer vision, machines are learning to see the world not just as a collection of colors and shapes, but as a list of distinct objects with names, locations, and boundaries. For years, the most successful way to teach a computer to do this has been to show it millions of examples and ask it to guess the next word in a description, much like a student memorizing a textbook. This method, known as supervised training, works well when there are only a few objects to find. However, it struggles when the scene becomes crowded. If a computer is asked to find every bird in a flock or every car in a traffic jam, the standard training method often causes the system to panic, repeating the same object over and over or giving up entirely before the list is finished. The core problem is that the training method optimizes for the right answer word-by-word, rather than ensuring the final list of objects is complete and accurate.

A team of researchers at the Technology Innovation Institute and the University of Tübingen has found a way to fix this by changing how the computer learns. Instead of just memorizing examples, they taught their model, called Falcon Perception-HD, to learn from its own mistakes through a process similar to trial and error. They used a technique called reinforcement learning, where the model generates a list of objects, checks how well that list matches the real scene, and receives a score. If the model misses an object or lists the same one twice, it gets a lower score and learns to adjust its behavior. By repeating this process, the model discovered a better way to count and locate things, effectively teaching itself to stop making the errors that plagued previous systems.

The results of this approach are most dramatic in the most difficult scenarios: scenes packed with hundreds of objects. In tests involving images with up to 500 items, older models would often collapse, failing to recognize more than a fraction of the scene or producing a chaotic jumble of duplicate detections. The new model, however, handled these dense environments with remarkable stability. It successfully identified nearly every object in the image, achieving a level of performance that previous systems could not reach. This is a significant leap because it allows computers to operate in complex, real-world environments like busy city streets or crowded ecosystems, where counting and locating every single item is essential.

One of the most surprising discoveries was that the model learned to clean up its own mess without needing external help. Traditionally, when a computer vision system produces a list of objects, it must run a separate, manual filtering step to remove duplicates and overlapping boxes. This step is fragile; if the settings are slightly off, it might delete a real object or keep a fake one. The researchers found that their model, after learning through reinforcement, naturally stopped producing duplicates in the first place. It learned to space out its predictions and stop generating new objects when the scene was full, effectively internalizing the rules that humans had to program into older systems. This means the system is not only more accurate but also faster and simpler to use, as it no longer relies on these error-prone manual filters.

The team also solved a subtle but critical problem regarding how the model decides if an object exists at all. In many real-world situations, a computer must determine whether a specific item is present or absent, such as checking if a particular animal is in a forest. Previous attempts to use this trial-and-error learning method often caused the model to become overly optimistic, guessing that an object was present even when it was not. The researchers found a way to prevent this by carefully controlling how the model's confidence was updated during training. This allowed the system to maintain a sharp distinction between "present" and "absent," ensuring that it did not start hallucinating objects that were not there.

To train the model, the researchers did not just rely on perfect human-made labels, which are expensive and time-consuming to create. Instead, they developed a smart pipeline where the model generated its own practice problems. They identified the moments where the model was uncertain or made mistakes and used those specific cases to teach it how to improve. This self-improvement loop allowed them to create a high-quality training set from a relatively small number of images. The result is a system that can handle extreme density, from a few objects to hundreds, with a single, unified approach.

The implications of this work extend beyond just better image recognition. By showing that reinforcement learning can fix fundamental flaws in how computers perceive the world, the researchers have opened a new path for building more robust artificial intelligence. The system they built, Falcon Perception-HD, sets a new standard for performance in crowded scenes, outperforming much larger models that rely on older training methods. It demonstrates that with the right learning strategy, machines can learn to see the world with a clarity and reliability that was previously out of reach, turning chaotic visual data into precise, actionable information.

Technical Summary: Falcon Perception-HD

Problem Statement

Autoregressive perception models, designed to localize visual entities in open-vocabulary settings, are traditionally trained via Supervised Fine-Tuning (SFT) using Maximum Likelihood Estimation (MLE). This approach optimizes a proxy objective (per-token cross-entropy) that is fundamentally misaligned with the order-invariant set-matching metrics used to evaluate perception (e.g., precision, recall, and F1). Consequently, MLE-trained models often suffer from:

Sequence Collapse: In dense scenes, models tend to stop generating tokens prematurely or produce degenerate sequences.
Redundancy: Models frequently generate duplicate predictions, overlapping masks, and repeated coordinates.
Heuristic Dependence: To mitigate these issues, existing systems rely on fragile post-processing heuristics like Non-Maximum Suppression (NMS) and coordinate deduplication, which require hyperparameter tuning and can suppress valid, overlapping objects.
Scalability Limits: Performance degrades sharply in high-density regimes (e.g., >200 objects per scene), where autoregressive generation becomes unstable.

Methodology

The paper introduces Falcon Perception-HD, a framework that applies Reinforcement Learning (RL) post-training to the autoregressive Falcon Perception model to directly align generation with perception metrics.

Core Architecture and Action Space

The base model employs a "chain-of-perception" architecture, emitting a repeating sequence for each object: text tokens (referring expression) $\to$ center coordinates $(x, y)$ $\to$ box size $(h, w)$ $\to$ segmentation mask.

Heterogeneous Action Space: The policy $\pi_\theta$ samples from different heads: a standard LM head for text, categorical heads for discretized coordinates and sizes, and a deterministic continuous head for masks.
Training Strategy: During RL, the authors sample rollouts from the LM and coordinate heads while decoding the size head greedily. The segmentation head remains frozen. This design leverages a "cascade effect," where improved spatial localization (coordinates) naturally conditions downstream heads to produce higher-quality sizes and masks without direct gradient updates.

RL Objective and Algorithm

Algorithm: The authors utilize GRPO (Group Relative Policy Optimization), which computes advantages relative to a group of $G=8$ rollouts sampled from the same prompt, eliminating the need for a separate value function.
Reward Function: A set-matching reward based on the Hungarian algorithm. It penalizes False Negatives (FN) and False Positives (FP) with a weighted count: $r_{count} = -(FN + \alpha \cdot FP)$ , where $\alpha=0.3$ . This reward is order-invariant and does not depend on the specific sequence order of generation.
Loss Aggregation: To prevent bias toward shorter responses (which is critical in dense scenes requiring long sequences), the authors adopt Dr. GRPO, normalizing losses by a fixed constant rather than the variable rollout length.
Engine Mismatch Correction: An importance sampling ratio with a stop-gradient is applied to correct discrepancies between the inference engine (used for rollouts) and the training engine.

Key Design Choices

Existence Token Detachment: The log-probabilities of existence tokens (<object_found> / <no_object_found>) are detached from the policy gradient. Since RL is trained only on positive queries, updating these tokens would cause the model to indiscriminately predict "found," collapsing the Matthews Correlation Coefficient (MCC) for existence calibration.
Self-Annotation Pipelines: Two hybrid pipelines curate training data:
- Dense Scenes: Human annotators refine pseudo-labels generated by the base model on images with >200 instances.
- Hard Referring Expressions: The model samples multiple rollouts; only cases where the greedy prediction fails but a sampled rollout succeeds are retained, ensuring high-entropy learning signals.

Key Contributions

Direct Metric Alignment: Demonstrates that RL can bridge the gap between autoregressive generation and set-matching evaluation metrics, resolving the misalignment inherent in MLE.
Elimination of Heuristics: The post-trained model structurally internalizes spatial suppression, effectively learning to avoid duplicate coordinates and mask repetitions. This removes the need for NMS and coordinate deduplication thresholds, improving both inference efficiency and robustness.
High-Density Robustness: The framework unlocks state-of-the-art performance in extreme-density regimes (up to 500 objects per scene), a domain where MLE-trained baselines typically collapse.
Cascade Effect Validation: Shows that optimizing only the discrete spatial heads (coordinates) is sufficient to improve the entire perception chain, including the frozen continuous segmentation head.
Preservation of Existence Calibration: By detaching existence tokens, the model maintains its ability to distinguish between scenes where an object exists and where it does not, without training on negative samples.

Experimental Results

Evaluated on PBench (referring expression segmentation) and SACO (open-vocabulary segmentation):

PBench: Falcon Perception-HD achieves a new state-of-the-art, improving the average macro-F1 by 2.6 points over the SFT baseline. Notably, on the Dense split (up to 500 objects), it improves by 8.2 points (72.3 $\to$ 80.5), outperforming significantly larger models (e.g., Qwen3-30B).
SACO: The model improves macro-F1 by 1.7 points and MCC by 2%, demonstrating better object discrimination and existence calibration.
Ablation Studies:
- Removing post-processing (NMS/dedup) causes the SFT baseline to lose 12.0 points in pmF1, whereas Falcon Perception-HD loses only 0.4 points.
- Sampling the size head in addition to coordinates yields no additional gain, confirming the cascade effect.
- Without the existence-token stop-gradient, MCC collapses from 59.6 to 38.8.

Significance and Claims

The paper claims that Falcon Perception-HD demonstrates that RL post-training is not merely a tool for aligning language models with human preferences but is a critical pathway for building robust perception models. By replacing fragile, inference-time heuristics with a policy that natively understands set-matching constraints, the authors show that autoregressive models can achieve stability and scalability in dense visual environments previously inaccessible to them. The work suggests that the "sequence collapse" and redundancy issues in autoregressive vision are algorithmic flaws of MLE training that can be resolved through a short, targeted RL stage, effectively acting as a learned Hungarian matching mechanism.