💻 computer science

RAWDet-7: A Multi-Scenario Benchmark for Object Detection and Description on Quantized RAW Images

The paper introduces RAWDet-7, a large-scale, multi-scenario benchmark featuring densely annotated quantized RAW images designed to evaluate object detection and description capabilities while studying information preservation under various bit-depth constraints.

Original authors: Mishal Fatima, Shashank Agnihotri, Kanchana Vaishnavi Gandikota, Michael Moeller, Margret Keuper

Published 2026-02-11

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Mishal Fatima, Shashank Agnihotri, Kanchana Vaishnavi Gandikota, Michael Moeller, Margret Keuper

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to solve a complex jigsaw puzzle, but instead of having the original, high-quality box lid to guide you, you only have a blurry, low-resolution photocopy of the picture. Even worse, someone has taken a thick black marker and scribbled over parts of the photo, or tried to "simplify" it by using only a few colors.

This is essentially the problem researchers face when trying to teach AI to "see" using RAW images.

Here is a breakdown of the RAWDET-7 paper using everyday analogies.

1. The Problem: The "Filtered" World vs. The "Real" World

Most AI models are trained on sRGB images. Think of sRGB as a "finished meal" prepared by a chef (the camera's internal software, called an ISP). The chef adds salt, spices, and presentation to make the food look delicious to humans. However, in the process, the chef might throw away the "raw ingredients"—the tiny details about the exact texture of the meat or the precise acidity of the sauce.

RAW images, on the other hand, are the "raw ingredients." They contain much more information, but they are messy, unorganized, and hard for a standard AI to digest. If we want AI to work in specialized ways—like in self-driving cars or high-tech sensors—we need it to understand the raw ingredients, not just the finished meal.

2. The Challenge: The "Low-Battery" Constraint

In many real-world devices (like a tiny drone or a smart doorbell), we don't have the luxury of massive computing power. We need to save energy and memory. To do this, we use Quantization.

Think of quantization like trying to paint a masterpiece using only a box of 4 crayons instead of a professional set of 1,000 paints. If you just pick 4 colors randomly, the picture looks terrible. But if you are smart about which 4 colors you pick, you can still make a recognizable image.

3. The Solution: RAWDET-7 (The Ultimate Training Manual)

The researchers created RAWDET-7, which is like a massive, super-accurate training manual for AI. It does three main things:

The Cleanup Crew: Previous datasets were like old, dusty textbooks with typos and missing pages (incorrect labels or missing objects). The researchers used a powerful AI to "re-read" and "re-write" the labels, making sure every car, person, and bicycle is correctly identified and located.
The Multi-Scenario Gym: They didn't just show the AI sunny days. They included night scenes, different types of camera sensors, and high-contrast lighting. It’s like training an athlete in the rain, the snow, and the heat so they are ready for anything.
The "Crayon" Test: They specifically tested how well AI performs when it's forced to use those "4, 6, or 8 crayons" (low-bit quantization).

4. The Big Discovery: "Smart Scaling"

The most exciting part of the paper is their discovery about how to handle those "limited crayons."

They found that if you use "Linear Scaling" (just picking colors blindly), the AI fails miserably. It’s like trying to paint a sunset with just blue, red, yellow, and black.

However, if they used "Learnable $\gamma$ -Scaling" (a fancy way of saying they let the AI learn exactly which "shades" are most important to keep), the results were incredible. They proved that an AI looking at a "4-crayon" RAW image can actually perform just as well, or even better, than an AI looking at a standard, high-quality color photo.

5. The "Storyteller" Test (Object Description)

Finally, they didn't just want the AI to say, "There is a car." They wanted to see if the AI could still "describe" the scene.

They tested if the AI could look at a low-quality, quantized image and still tell a detailed story (e.g., "A silver sedan is parked on a wet road with reflections of streetlights on its hood"). They found that by using their "smart scaling" method, the AI's descriptions remained remarkably detailed and accurate, even when the image data was heavily compressed.

Summary in a Nutshell

RAWDET-7 is a massive, high-quality toolkit that teaches AI how to see the world through "raw ingredients" rather than "processed meals." It proves that even if we limit an AI's "brainpower" or "color palette" to save energy, we can still achieve world-class vision—as long as we teach it how to pick the right details to focus on.

Technical Summary: RAWDET-7: A Multi-Scenario Benchmark for Object Detection and Description on Quantized RAW Images

1. Problem Statement

Most computer vision models are trained on standard 8-bit sRGB images. These images are the product of an Image Signal Processor (ISP) pipeline designed for human aesthetics (e.g., denoising, color correction, gamma adjustment), which often discards high-precision sensor-level data essential for machine reasoning. While RAW images preserve a much higher dynamic range and bit depth (up to 24-bit), research in this area is bottlenecked by:

Lack of Large-Scale Datasets: Existing RAW datasets are small, have limited object classes, or suffer from poor annotation quality (hallucinations and misclassifications).
Quantization Constraints: In edge computing and low-power sensing, reducing bit depth (quantization) is necessary to save memory and power, but the impact of heavy quantization on complex downstream tasks like object detection and description is poorly understood.
ISP Bias: Most modern detectors are optimized for sRGB, making it unclear how they perform on raw sensor data without a traditional ISP.

2. Methodology

The authors propose RAWDET-7, a comprehensive benchmark designed to bridge the gap between raw sensor data and high-level vision tasks.

A. Dataset Construction:

Consolidation: The authors unified four existing datasets (PASCAL RAW, NOD, RAOD, and Zurich) to create a large-scale corpus of $\sim$ 32k RAW images ( $\sim$ 25k for training, $\sim$ 7.6k for testing).
Re-annotation: To fix errors in previous datasets, they used Grounded-DINO 1.5 to re-annotate the images for seven specific categories (Car, Truck, Tram, Person, Bicycle, Motorcycle, Bus) following MS-COCO/LVIS conventions. They applied a strict confidence threshold ( $>0.8$ ) and human validation to ensure high-quality, dense labels.
Object Description Task: To evaluate semantic preservation, they used Gemini-2.5-Pro to generate detailed, object-level captions for a subset of high-resolution sRGB images, creating a "ground truth" reference for description quality.

B. Benchmarking Framework:

Quantization Simulation: The dataset allows evaluation under simulated 4-bit, 6-bit, and 8-bit quantization.
Scaling Methods: They tested four input scaling strategies to mitigate the information loss of quantization:
1. Linear Scaling: Naïve normalization.
2. Logarithmic Scaling: Non-linear dynamic range compression.
3. $\gamma$ -Scaling: A learnable non-linear mapping optimized alongside the neural network.
4. Log + $\gamma$ -Scaling: A hybrid approach combining logarithmic and learned $\gamma$ mappings.
Models Evaluated: They tested traditional two-stage/one-stage detectors (Faster R-CNN, RetinaNet, PAA) and large foundation models (MM-Grounding DINO).

3. Key Contributions

RAWDET-7 Dataset: A large-scale, multi-scenario, high-quality RAW dataset covering diverse lighting (day/night), sensors (Sony, Nikon), and bit depths.
New Task Dimension: Introduced object-level description as a metric to evaluate how much fine-grained detail is preserved in processed/quantized RAW images.
Quantization-Aware Benchmarking: Provided a systematic study of how different scaling methods affect detection and description performance in low-bit regimes.

4. Results

Detection Performance:
- Naïve linear quantization performs poorly, especially at 4-bit.
- Log + $\gamma$ -Scaling significantly outperforms other methods. Notably, models using these scaling methods on 6-bit or 8-bit RAW data can perform on-par with or even surpass models trained on standard 8-bit sRGB images.
- Even large foundation models (MM-Grounding DINO) show improved performance when using $\gamma$ -scaling, even when the model itself is kept frozen.
Annotation Quality: User studies confirmed that RAWDET-7 annotations are significantly more accurate and dense than previous datasets, resolving issues of hallucinations and missing small/occluded objects.
Description Fidelity: Using Gemini as a judge, the authors demonstrated that Log + $\gamma$ quantized images produce object descriptions that closely match the semantic similarity and detail levels of high-resolution sRGB references, whereas linear quantization fails to provide meaningful descriptions.

5. Significance

This work is highly significant for the development of next-generation intelligent sensors. It proves that we do not necessarily need a heavy, human-centric ISP pipeline for machine vision; instead, we can use lightweight, task-specific non-linear mappings (like $\gamma$ -scaling) to process low-bit RAW data. This enables the design of highly efficient, low-power vision systems for edge devices (e.g., autonomous drones, IoT sensors) that can maintain high-level reasoning capabilities while operating on significantly reduced bit-depth data.