💻 computer science

FOCUS: Bridging Fine-Grained Recognition and Open-World Discovery across Domains

The paper introduces FoCUS, the first unified framework for Fine-Grained Domain-Generalized Generalized Category Discovery (FG-DG-GCD), which addresses the challenge of recognizing known and discovering novel classes across unseen domains by combining domain-consistent part discovery with uncertainty-aware feature augmentation, while also establishing new benchmarks and demonstrating superior accuracy and computational efficiency over existing baselines.

Original authors: Vaibhav Rathore, Divyam Gupta, Moloud Abdar, Subhasis Chaudhuri, Biplab Banerjee

Published 2026-03-17

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Vaibhav Rathore, Divyam Gupta, Moloud Abdar, Subhasis Chaudhuri, Biplab Banerjee

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are teaching a child to recognize different types of birds.

In a normal classroom (the "closed world"), you show the child pictures of a Blue Jay, a Cardinal, and a Sparrow. You say, "This is a Blue Jay," and they learn to spot the blue feathers. When you show them a new picture of a Blue Jay, they get it right.

But the real world is messy. This is the "Open World" problem.

The Style Shift: Suddenly, you show the child a picture of a Blue Jay, but it's drawn in a sketch style, or painted in oil, or looks like a watercolor. The colors are gone, the textures are different. The child, who only learned from photos, gets confused. "Is this still a Blue Jay? It looks like a pencil drawing!"
The New Mystery: Then, you show the child a picture of a Puffin. They've never seen a Puffin before. A standard AI (or a child who only memorized the list) would either say, "I don't know," or worse, it would force the Puffin into the "Blue Jay" category because it's the closest thing it knows.

This paper introduces FOCUS, a new way to train AI to handle both of these problems at once, especially for "fine-grained" tasks where the differences are tiny (like telling a 200th species of bird apart from a 199th).

Here is how FOCUS works, using simple analogies:

1. The Problem: Why Current AI Fails

Most AI models are like students who memorize the texture of the paper rather than the shape of the object.

If you train an AI on photos of cars, it learns "shiny metal" and "tires."
If you show it a sketch of a car (no metal, just lines), the AI panics because the "shiny metal" is gone.
If you show it a new type of car it's never seen, it tries to shove it into the "Toyota" box because it's too scared to admit it doesn't know.

2. The Solution: FOCUS

The authors created a framework called FOCUS (Fine-grained Open-world Category discovery with Uncertainty and partS). It has two main superpowers:

Superpower A: The "Skeleton Goggles" (Domain-Consistent Parts Discovery)

Imagine you are trying to recognize a friend in a crowd, but they are wearing a different costume, and the lighting is weird. If you look at their skin tone or hair color, you might get confused. But if you look at their skeleton (the shape of their nose, the curve of their jaw, the way their ears stick out), you will recognize them instantly.

How FOCUS does it: Instead of looking at the "paint" (colors and textures), FOCUS puts on "Skeleton Goggles." It ignores the style (whether it's a photo, a sketch, or a painting) and focuses entirely on the geometry (the lines, the curves, and the structure).
The Result: It can tell that a "sketch of a bird" is the same bird as a "photo of a bird" because the beak shape and wing structure are identical, even if the colors are totally different.

Superpower B: The "Honesty Meter" (Uncertainty-Aware Feature Augmentation)

Imagine a student taking a test. When they see a question they don't know, a bad student guesses, "It's probably a Blue Jay!" and marks it confidently. A smart student says, "I don't know what this is, but I'm sure it's not a Blue Jay."

The Problem: Old AI models are terrible at admitting they are wrong. They get "overconfident" and force new, unknown things into old categories.
How FOCUS does it: FOCUS has an internal "Honesty Meter." During training, it creates fake, weird examples (like a blurry mix of two birds) and teaches the AI: "When you see something like this, you must say 'I am unsure.' Do not guess!"
The Result: When the AI sees a brand new bird (a Puffin) in a sketch, instead of forcing it into the "Blue Jay" box, it says, "This is new! Let's make a new box for it."

3. The "Training Gym" (The Benchmarks)

To prove this works, the authors couldn't just use normal photos. They needed to test the AI in a "style-shift" gym.

They took famous bird, car, and plane datasets.
They used a special AI tool (like a digital artist) to turn every photo into a painting and a sketch without changing the shape of the object.
They trained the AI on the photos and tested it on the paintings and sketches.

The Bottom Line

FOCUS is like a detective who learns to ignore the suspect's clothes (which change with the weather or location) and focuses on their bone structure (which never changes).

It is faster: It doesn't need to generate thousands of fake images to learn; it learns the "skeleton" directly.
It is smarter: It knows when it doesn't know something, so it can discover new categories instead of making mistakes.
It is robust: It works whether the image is a photo, a sketch, or a painting.

In short, FOCUS teaches AI to see the essence of an object, not just its appearance, allowing it to learn new things even when the world changes its style.

1. Problem Definition: FG-DG-GCD

The paper addresses a critical gap in computer vision by formalizing a new task called Fine-Grained Domain-Generalized Generalized Category Discovery (FG-DG-GCD). This setting combines three challenging objectives that existing methods fail to solve simultaneously:

Fine-Grained Recognition: Distinguishing between visually similar categories (e.g., bird species or car models) where differences are subtle and localized.
Domain Generalization (DG): The model is trained only on labeled data from a source domain but must perform on an unseen target domain with a different distribution (e.g., shifting from real photos to sketches or paintings).
Generalized Category Discovery (GCD): The model must not only recognize known classes but also discover and cluster novel (unseen) classes in the target domain without any target-side supervision.

The Core Challenge: In fine-grained settings, subtle inter-class cues (like beak shape or wing structure) are easily suppressed by domain shifts (texture, lighting, style). Existing methods either fail to generalize across domains or collapse novel classes into known manifolds because they rely on appearance-dependent features (color/texture) rather than geometry-stable cues.

2. Methodology: The FOCUS Framework

The authors propose FOCUS (Fine-grained Open-world Category discovery with Uncertainty and partS), a single-stage, synthetic-free, and episodic-free framework. It consists of two core modules:

A. Domain-Consistent Parts Discovery (DCPD)

To handle domain shifts without relying on texture, DCPD focuses on geometry-stable part reasoning.

Image-Conditioned Queries: Unlike previous part-based methods (e.g., APL) that use static, globally shared queries prone to source-domain bias, DCPD uses image-conditioned part queries. These queries are dynamically grounded in a geometry-stable ViT attention prior.
Self-Attention Priors: The module extracts coarse geometric priors by thresholding ViT self-attention maps to identify structural regions (contours, edges) that remain invariant across styles.
Differentiable Assignment: It employs a Straight-Through Gumbel-Softmax mechanism to assign image patches to specific parts. This enforces "structural exclusivity," preventing feature bleeding and ensuring parts remain spatially coherent.
Part-Global Fusion: DCPD fuses these localized part features with the global [CLS] token. This ensures the model retains holistic context while anchoring decisions on domain-invariant structural evidence.

B. Uncertainty-Aware Feature Augmentation (UFA)

To prevent the model from being overconfident about novel classes (a common failure mode in open-world settings), UFA performs calibrated open-space regularization directly in the embedding space.

Tripartite OOD Sampling: Instead of generating synthetic images (which can distort fine-grained details), UFA synthesizes feature perturbations in the embedding space using three strategies:
1. Tail-Variation Sampling: Samples near the boundaries of known class manifolds to simulate rare instances.
2. Between-Class Ambiguity Mixing: Samples in the transitional zones between class centroids to model semantic ambiguity.
3. Far-Away Unknown Sampling: Samples from the hypersphere to represent completely unknown regions.
Regularization: These synthetic outliers are used to train the classifier via Energy-Based Outlier Exposure (OE) and Entropy Maximization. This forces the model to assign high energy and high entropy to unknown samples, creating a well-calibrated decision boundary that prevents novel classes from collapsing into known clusters.

3. Key Contributions

Problem Formalization: Defined FG-DG-GCD, a unified framework for fine-grained recognition, domain generalization, and novel class discovery, highlighting the unique dual challenge of preserving precision under shift while maintaining structural flexibility for discovery.
New Benchmarks: Constructed the first dedicated benchmarks for this task using CUB-200-2011, Stanford Cars, and FGVC-Aircraft. They utilized a controlled SDXL-adapter pipeline to generate identity-preserving "Painting" and "Sketch" domains. This ensures geometric structure is preserved while inducing massive stylistic shifts, avoiding the semantic corruption common in naive style transfer.
The FOCUS Framework: Introduced a unified, lightweight architecture that outperforms existing baselines without relying on expensive domain synthesis or episodic training.
Efficiency: The method achieves state-of-the-art results with ~3× higher computational efficiency compared to current SOTA methods (like HIDISC) and nearly 300× lighter than others (like DG2CD-Net).

4. Experimental Results

The authors evaluated FOCUS on the proposed multi-domain fine-grained benchmarks and standard coarse-grained DG benchmarks (PACS, Office-Home, DomainNet).

Fine-Grained Performance:
- FOCUS achieved an average All Accuracy of 38.01% on the fine-grained benchmarks, outperforming the best prior baselines by +1.17% (GCD) and +1.95% in New Class discovery.
- It significantly outperformed specialized Fine-Grained GCD (FG-GCD) and Domain Generalization (DG) baselines, which struggled with the combined constraints.
- Ablation Studies: Confirmed that removing either DCPD or UFA causes significant performance drops. The synergy between geometry-stable parts and uncertainty calibration is critical.
Coarse-Grained Performance:
- Despite being designed for fine-grained tasks, FOCUS achieved competitive or superior results on coarse-grained benchmarks (e.g., 74.35% on PACS), demonstrating the robustness of its geometry-invariant representations.
Cluster Estimation: FOCUS showed superior ability to estimate the number of novel clusters ( $K$ ), with the lowest error compared to ground truth, whereas other methods significantly overestimated the number of classes due to domain-induced fragmentation.
Qualitative Analysis: t-SNE visualizations showed that FOCUS produces compact, well-separated clusters, whereas baselines exhibited scattered or overlapping manifolds.

5. Significance and Impact

Bridging the Gap: This work is the first to successfully unify fine-grained discrimination, domain generalization, and open-world discovery, addressing a realistic scenario where systems must operate in changing environments with unknown categories.
Efficiency vs. Performance: By avoiding heavy synthetic data generation (diffusion/GANs) and episodic meta-learning, FOCUS offers a highly efficient solution that is more scalable for real-world deployment.
Geometry over Texture: The paper establishes that for fine-grained open-world learning, relying on geometric priors (structure/contours) rather than appearance priors (texture/color) is essential for robustness against domain shifts.
Resource Availability: The authors released the code and the new identity-preserving painting/sketch datasets, facilitating further research in robust open-world recognition.

In summary, FOCUS provides a robust, efficient, and theoretically grounded solution for deploying fine-grained visual recognition systems in dynamic, open-world environments where both the domain and the set of categories are unknown.