🤖 machine learning

Know Yourself Better: Diverse Object-Related Features Improve Open Set Recognition

This paper analyzes the role of feature diversity in open set recognition, revealing its strong correlation with performance and proposing a novel method that leverages diverse discriminative features to significantly outperform state-of-the-art approaches.

Original authors: Jiawen Xu, Margret Keuper

Published 2026-06-05

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Jiawen Xu, Margret Keuper

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Problem: The "Overconfident Guess"

Imagine you are a security guard at a club. You have a list of VIPs (the "known" classes) you are trained to recognize: Alice, Bob, and Charlie.

One night, a stranger walks in who looks a bit like Alice. Because your training only covered Alice, Bob, and Charlie, your brain tries to force a match. You confidently say, "That's Alice!" even though it's actually a stranger. In the world of AI, this is called Open Set Recognition (OSR). The problem is that standard AI models are terrible at saying, "I don't know who this is." They just guess the closest thing they know, often with 100% confidence, which is dangerous in real life.

The Core Idea: Don't Just Look at One Thing

The authors of this paper asked a simple question: Why do these models fail to spot strangers?

They found that the models are too lazy. They learn to recognize things using just one obvious feature.

The Analogy: Imagine you are teaching a child to recognize a cat. If you only show them pictures of cats with orange fur, the child might learn that "Orange = Cat."
The Failure: If a leopard (a stranger) walks in wearing an orange coat, the child screams, "It's a cat!" because they only looked at the color. They ignored the ears, the tail, and the spots.

The paper argues that to spot strangers (outliers), a model needs to learn many different features at once (color, shape, texture, size), not just the easiest one. This is called Feature Diversity.

The Experiment: The "Shape vs. Color" Test

To prove this, the researchers ran a simple game with a computer:

Scenario A: They showed the computer blue circles and red rectangles. The computer easily learned to tell them apart by just looking at color (Blue vs. Red).
Scenario B: They added a red circle to the mix. Now, color wasn't enough. To tell a red circle from a red rectangle, the computer had to learn about shape too.

The Result: The computer in Scenario B (which learned both color and shape) was much better at spotting a "stranger" (a blue rectangle) than the computer in Scenario A (which only knew color).

Lesson: When a model is forced to learn more diverse features to do its job, it becomes better at spotting things it doesn't know.

The Secret Sauce: The "Temperature" Knob

The researchers then looked at a specific tool used to train AI called Supervised Contrastive Learning (SupCon). This tool helps the AI group similar things together and push different things apart.

Inside this tool, there is a setting called Temperature (think of it like a volume knob or a spice level).

Low Temperature: The AI gets very strict. It focuses intensely on the hardest-to-distinguish pairs. It says, "I must find the tiny differences between these two!"
High Temperature: The AI is more relaxed. It focuses on the easy differences.

The Discovery: The researchers found that different "temperature" settings make the AI look at the data in completely different ways.

One model might focus on the texture of a cat's fur.
Another model (trained at a different temperature) might focus on the shape of its ears.

They are like two detectives looking at the same crime scene. Detective A looks at the footprints; Detective B looks at the fingerprints. Neither has the whole picture alone, but together, they solve the case.

The Solution: The "Teamwork" Approach

Since one model trained at one temperature only sees part of the picture, the authors proposed a simple solution: The Ensemble.

Instead of training one AI model, they train three models, each with a different "temperature" setting.

Model 1 (Low Temp) looks at the fine details.
Model 2 (Medium Temp) looks at the medium details.
Model 3 (High Temp) looks at the big picture.

When a new image comes in, they ask all three models for their opinion and combine the answers. Because they are looking at the object from different angles (using different features), they are much better at spotting the "strangers" that a single model would miss.

The Results

The team tested this on standard image datasets (like CIFAR-100 and TinyImageNet).

The Outcome: Their "Teamwork" approach consistently beat other top methods.
Why? It wasn't because they used a fancier algorithm. It was simply because they forced the models to learn a wider variety of features by using different temperatures, making the final decision much more robust.

Summary

The paper teaches us that to make AI safer and better at recognizing the unknown, we shouldn't just train it to be a master of one trick. We should train it to be a generalist that notices many different details. By combining the "opinions" of models trained with different settings, we get a system that knows its own limits and is much less likely to confidently guess the wrong answer.

Technical Summary: Know Yourself Better: Diverse Object-Related Features Improve Open Set Recognition

Problem Statement

Conventional neural classifiers operate under a "close-set" assumption, meaning they are trained on a fixed set of categories and struggle to identify samples from previously unseen classes. In realistic deployment scenarios, this leads to overconfident predictions for "open set" samples (outliers), which are misclassified as known "inlier" classes. Open Set Recognition (OSR) aims to solve this by enabling models to detect novel classes at inference time. While various methods exist, the underlying representation-learning mechanisms that drive successful OSR remain insufficiently understood. Specifically, the role of feature diversity—the extent to which a representation captures multiple semantically meaningful object attributes rather than relying on a limited subset of discriminative cues—has received limited attention.

Methodology

1. Investigation of Feature Diversity

The authors first establish a link between feature diversity and OSR performance through controlled experiments:

Synthetic Experiments: Using a dataset of circles and rectangles, they trained models on scenarios requiring either color-only discrimination (E1) or combined color and shape discrimination (E2). Models trained on the more complex E2 scenario (requiring shape features) demonstrated superior OSR performance when tested against outliers (blue rectangles) that shared color cues with inliers but differed in shape.
CIFAR-100 Experiments: By training classifiers on increasing numbers of classes (from coarse macro-classes to fine-grained classes), the authors observed that models required to distinguish more fine-grained classes learned more diverse features. These models consistently achieved higher OSR performance (measured by AUROC) and better generalization to outlier classes compared to models trained on fewer classes.

2. Analysis of Supervised Contrastive Learning (SupCon)

The paper investigates the temperature parameter ( $\tau$ ) in Supervised Contrastive Learning (SupCon).

Gradient Analysis: The authors analyze the gradients of the SupCon loss with respect to positive and negative pair similarities. They demonstrate that the temperature parameter significantly influences the optimization dynamics:
- Low temperatures impose stronger attraction on positive pairs and assign larger gradients to "hard" negative pairs (samples with high similarity but different labels). This forces the model to learn more invariant and discriminative characteristics.
- High temperatures assign relatively greater gradients to easier and moderately difficult negatives.
Complementary Representations: Empirical analysis shows that models trained with different temperatures learn representations that are less aligned (lower prediction agreement) and capture complementary aspects of the data. Aggregating representations from models trained with different temperatures yields higher classification accuracy on in-distribution data than using a single model.

3. Proposed Approach: Temperature-Based Ensemble

Motivated by the finding that different temperatures induce complementary feature learning, the authors propose a simple ensemble framework:

Training: Multiple SupCon models are trained independently using different temperature values (sampled from a logarithmic distribution: $\{1, 0.5, 0.1, 0.05, 0.01, 0.005\}$ ).
Inference: During OSR inference, the models generate scores (e.g., feature norms or distances). These scores are aggregated (specifically, Score Summation was found to be the most effective strategy among concatenation, summation, and score summation) to produce a final outlier detection score.
Architecture: The method uses a standard ResNet-18 backbone without pre-training or foundation models, relying purely on the diversity induced by the temperature hyperparameter.

Key Results

Extensive experiments were conducted on six standard OSR benchmarks: MNIST, SVHN, CIFAR-10, CIFAR+10, CIFAR+50, and TinyImageNet.

Performance: The proposed method consistently outperformed strong baselines, including OpenMax, C2AE, APRL, and GeoEnsemble.
State-of-the-Art: The approach achieved the best or comparable results across most protocols. Notably, it achieved an AUROC of 81.0% on the complex TinyImageNet dataset, surpassing previous methods like SupCon-ST (80.9%) and MEDAF (80.0%).
Ablation Studies:
- Aggregating three representations from different temperatures yielded the best performance (e.g., 93.12% on CIFAR-10), outperforming single or double representations.
- The choice of temperature combinations significantly impacted performance, confirming that different temperatures capture distinct feature subsets.
- Score summation (SocSum) proved slightly more robust than representation concatenation or summation.

Significance and Claims

The paper claims that the primary contribution is a shift in perspective from specific outlier detection strategies to the representation learning process itself.

Feature Diversity Hypothesis: The work provides empirical evidence that encoding a broader range of object-related features (diversity) is positively correlated with improved OSR performance.
Temperature as a Control Knob: The authors demonstrate that the temperature parameter in SupCon is not merely a hyperparameter for convergence but a mechanism to control the emphasis on hard vs. easy sample pairs, thereby shaping the diversity of the learned representations.
Simplicity and Effectiveness: The proposed solution is a simple ensemble of existing SupCon models with varying temperatures. It requires minimal modification to standard frameworks yet achieves competitive or superior performance, suggesting that leveraging representation diversity is a highly effective and underutilized strategy for OSR.
Broader Implications: While focused on OSR, the authors note that the principles of inducing feature diversity through representation learning may benefit other tasks such as transfer learning, model robustness, and domain adaptation.

The paper concludes that while foundation models offer broad capabilities, the specific principles of shaping representation learning to capture diverse, complementary features remain a critical and complementary research direction for reliable uncertainty quantification and outlier detection.