FDIM: A Feature-distance-based Generic Video Quality Metric for Versatile Codecs
The paper proposes FDIM, a hybrid deep and hand-crafted feature-based video quality metric trained on a large-scale dataset that achieves strong generalization and high correlation with subjective assessments across diverse traditional and neural codecs for both SDR and HDR content.
Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
Imagine you are a film critic trying to judge the quality of a movie. In the past, movies were made with traditional cameras and editing tools (traditional codecs). If something went wrong, the errors were predictable: the picture might get blocky, blurry, or have a weird "ringing" effect around edges. Old quality-checking tools were like critics who only knew how to spot these specific, predictable mistakes.
But now, a new kind of movie-making has arrived: Neural Video Codecs. These use artificial intelligence to compress video. Instead of just shrinking the file, the AI sometimes "hallucinates" new details to make the picture look smoother or sharper. While this often looks great, it creates weird, unpredictable errors—like textures that look too smooth, structures that drift slightly, or fake details that weren't there before. The old critics (traditional quality metrics) get confused by these new AI-made mistakes and often give the wrong score.
Enter FDIM: The "Hybrid Critic"
The authors of this paper created a new tool called FDIM (Feature-distance-based Generic Video Quality Metric). Think of FDIM as a super-critic that combines two different ways of judging a movie to handle both old-school and AI-made videos.
1. The Two Brains of FDIM
FDIM doesn't rely on just one way of thinking. It has two "brains" working together:
- Brain A (The Deep Learning Expert): This is a neural network trained on thousands of videos. It's like a critic who has watched every movie ever made and learned to spot subtle, complex errors. It can see when an AI codec has "hallucinated" a texture or smoothed out a face too much. It looks at the video from many different levels of detail, from tiny pixels to big-picture structures.
- Brain B (The Hand-Crafted Expert): This part uses established, rule-based tools (specifically VMAF). It's like a veteran critic who knows the classic rules of film quality. It's very good at spotting the old-school errors like blockiness or blurriness that the AI brain might sometimes miss or over-correct.
The Magic: FDIM takes the scores from both brains, averages them out, and gives a final verdict. This "hybrid" approach ensures it doesn't get confused by either old-school blockiness or new-school AI weirdness.
2. How It "Sees" the Errors
The paper describes a clever trick the Deep Learning brain uses called Content-Adaptive Feature-Distance Modeling.
Imagine you are comparing two paintings. If you just measure the distance between every single brushstroke, you might get it wrong if the paintings are slightly shifted or if the lighting is different.
- The Old Way: "This pixel is red, that one is blue. Bad!"
- FDIM's Way: It looks at the structure of the image first. If the original image has a sharp edge (like a tree branch), FDIM knows to look closely at that specific area to see if the AI messed it up. If the area is smooth (like a blue sky), it relaxes its focus. It's like a detective who knows exactly where to look for clues based on the context of the scene, rather than checking every inch of the floor randomly.
3. The Training Ground
To teach this critic, the authors didn't just use a few movies. They built a massive training set called DCVQA, containing over 16,000 video sequences.
- Some videos were compressed with old-school tools (like H.265).
- Some were compressed with new AI tools (Neural Codecs).
- Some were standard definition (SDR), and some were high dynamic range (HDR)—which is like comparing a black-and-white photo to a vibrant, 4K HDR photo.
By training on this huge, diverse mix, FDIM learned to be "general." It didn't just memorize one type of error; it learned the feeling of what makes a video look good or bad to a human eye.
4. The HDR Challenge (The "Zero-Shot" Trick)
One of the hardest things in video quality is judging HDR (High Dynamic Range) videos when you've only trained on SDR (Standard Dynamic Range) videos. It's like trying to judge a color movie after only watching black-and-white films.
Usually, you'd need to retrain the whole critic from scratch. But FDIM uses a clever preprocessing step called PU21. Think of this as a "translator" that converts the HDR video into a format that the SDR-trained critic can understand without needing to relearn everything.
- The Result: FDIM can look at a brand-new HDR video it has never seen before and give a surprisingly accurate score, just by using this translator. This is called "zero-shot transfer."
5. Does It Work?
The authors tested FDIM against many other popular tools on ten different datasets, including videos with codecs the tool had never seen before.
- The Verdict: FDIM consistently outperformed the competition. It was better at guessing how humans would rate the video quality, whether the video was made by an old codec, a new AI codec, or was in HDR.
- Efficiency: It's also fast enough to be useful in the real world, striking a good balance between being super smart and not taking forever to run.
Summary
In short, FDIM is a new video quality checker that combines the pattern-spotting power of AI with the reliable rules of traditional engineering. It was trained on a massive, diverse library of videos so it can handle everything from old-school blocky errors to new-school AI hallucinations, and it can even judge high-end HDR videos without needing special retraining. It's a versatile tool designed to keep up with the rapidly changing world of video technology.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.