Structural Regularities of Cinema SDR-to-HDR Mapping in a Controlled Mastering Workflow: A Pixel-wise Case Study on ASC StEM2

This paper presents an empirical case study using the ASC StEM2 dataset to characterize the structural regularities of cinema SDR-to-HDR mapping, revealing a stable global luminance correspondence and specific color redistribution patterns that allow for the classification of 82.4% of image regions as EXR-closer recoveries, thereby establishing a quantitative baseline for structure-aware analysis and learning-based model design.

The Gist

Imagine you have a very expensive, high-resolution photograph of a sunset. Now, imagine you want to print that photo on two different types of paper:

1. The "Standard" Paper (SDR): This is like the old, standard TV screens. It can only show a limited range of brightness. The brightest parts (the sun) get washed out into a white blob, and the darkest parts (shadows) turn into a muddy black.
2. The "Premium" Paper (HDR): This is like the new, fancy cinema projectors. It can show blindingly bright suns and deep, detailed shadows simultaneously.

For a long time, people thought converting the "Standard" photo to the "Premium" one was like reverse-engineering a recipe. They assumed that if you just used a smart computer algorithm, you could figure out exactly what the original sun looked like and "un-compress" the white blob back into a real sun.

This paper says: "Not so fast."

The researchers took a special test movie called The Mission (ASC StEM2). This movie is unique because they have the original raw footage (the "truth"), the Standard version, and the Premium version, all made from the same exact source. This is like having the original clay sculpture, the plaster cast, and the gold-plated statue, all made by the same artist.

Here is what they found, explained simply:

1. The "Global Rule" (The Elevator Analogy)

When they looked at the brightness of every single pixel in the movie, they found a very predictable pattern.

• The Analogy: Imagine an elevator. If you press "Floor 1" on the Standard version, it goes to "Floor 10" on the Premium version. If you press "Floor 2," it goes to "Floor 20."
• The Finding: The relationship between the Standard and Premium versions is almost perfectly monotonic (it always goes up, never down). It's a smooth, predictable slide.
• The Takeaway: You don't need a magic AI to guess the brightness. The Premium version is just a "stretched out" version of the Standard one. The structure of the image (where the edges and textures are) stays exactly the same.

2. The "Exceptions" (The Spotlight and the Glass)

However, the relationship isn't perfectly smooth everywhere. There are two specific places where the Premium version does something different than just stretching the Standard one:

• Self-Luminous Highlights (The Sun/Sparks): In the Standard version, a bright spotlight might just be a flat white circle because the screen can't handle the brightness. In the Premium version, the artists didn't just "un-compress" it; they actually added new detail to make it look like a real, glowing light source.
• Material Textures (Glass/Metal): Sometimes, the Premium version tweaks the reflection on a glass window or a metal car hood to make the material look more realistic, even if the Standard version looked a bit dull.
• The Takeaway: The "Premium" version isn't just a mathematical reverse-engineering of the Standard one. It's a creative adjustment for specific, tricky parts of the image.

3. The "Color" (The Paint Mixer)

What about colors?

• Hue (The Color Name): The researchers found that the "color names" (like "red" or "blue") stayed almost exactly the same. A red car in the Standard version is still a red car in the Premium version. The artists didn't want to change the story or the mood.
• Saturation (The "Pop"):
  • Shadows: In dark areas, the Premium version actually made colors less saturated (duller). Why? Because when you make shadows darker and more detailed, our eyes naturally perceive the colors as less intense.
  • Mid-tones: In normal lighting, the Premium version made colors slightly "pop" more.
  • Highlights: In very bright areas, colors got duller again because bright light naturally bleaches out color (think of how a bright white light makes a red wall look pink).

4. The "Truth Detector" (The Decision Map)

The most interesting part is how they decided: Is the Premium version just recovering the original truth, or is it making artistic changes?

They used the Original Raw Footage (EXR) as the "Truth Anchor."

• The Result: They found that 82% of the movie is just the Premium version faithfully recovering the original details that were squashed in the Standard version.
• The 18%: The other 18% is where the artists made smart, localized adjustments. They didn't try to "guess" the original sun; they painted a new, better-looking sun because the original data was lost or clipped.

The Big Conclusion

The paper argues that we shouldn't treat SDR-to-HDR conversion as a simple "math problem" where we try to physically reverse the compression.

Instead, think of it like restoring an old painting:

• 80% of the time: You are just cleaning the dirt off to reveal the original colors that were always there (Restoration).
• 20% of the time: You have to use your artistic judgment to repaint the parts that were damaged beyond repair, making them look better than the original damage, but still fitting the style of the painting (Creative Adaptation).

Why does this matter?
If you are building AI to convert old movies to HDR, don't just train it to be a "math wizard." Train it to be a smart restorer. It should know that for most of the image, it just needs to stretch the brightness smoothly. But for the bright lights and shiny surfaces, it needs to know when to stop guessing and start making creative, localized improvements.

Technical Summary

1. Problem Statement

The paper addresses the gap in understanding the structural relationship between professionally mastered Standard Dynamic Range (SDR) and High Dynamic Range (HDR) cinema releases.

• Current Limitations: Existing Inverse Tone Mapping (ITM) methods often assume SDR is a compressed physical record that can be algorithmically "inverted" to recover the original scene radiance. However, cinema masters are creative products involving exposure control, color grading, and narrative stylization.
• The Gap: There is a lack of systematic, quantitative evidence regarding whether a stable mapping exists between SDR and HDR masters derived from the same workflow, and how their structural differences (luminance and color) behave statistically.
• Goal: To establish a measurable baseline for SDR-to-HDR structural regularities within a shared-source mastering environment, moving beyond theoretical inversion to data-driven analysis of actual production workflows.

2. Methodology

The study utilizes ASC StEM2 ("The Mission"), a rare, authoritative test film designed for next-generation digital cinema systems.

• Data Source: The dataset includes three co-registered representations from a single ACES-based mastering pipeline:
  1. EXR Source: Scene-referred linear data (ACES AP0, 16-bit float).
  2. SDR Master: Display-referred (DCI-P3, Gamma 2.6, 48 cd/m²).
  3. HDR Master: Display-referred (DCI-P3, PQ, 300 cd/m²).
• Scope: Analysis covers all 18,580 frames of the film, supplemented by shot-level statistics (204 shots) and decision-map analysis on 91 representative frames.
• Analytical Framework:
  • Luminance Analysis: Used Isotonic Regression to fit a global monotonic mapping function ($f$) between SDR ($L_S$) and HDR ($L_H$) to test for order preservation. Residuals ($\Delta L = L_H - \hat{f}(L_S)$) were analyzed in a 2D "Energy-Structure" space using K-Means clustering.
  • Color Analysis: Performed in the perceptually uniform ICtCp color space. Metrics included Hue Stability ($\Delta h$), Chroma Correlation, and Saturation Enhancement Ratios across luminance bins.
  • Decision Mapping: A pixel-level decision map was constructed by comparing the perceptual distance ($\Delta E_{ITP}$) of both SDR and HDR against the EXR source. Pixels closer to EXR were classified as "EXR-closer recovery," while others were "content-adaptive adjustments."

3. Key Contributions

1. Three-Domain Comparative Framework: Established a rigorous comparison between EXR (scene-referred), SDR, and HDR masters within a single controlled ACES workflow.
2. Global Monotonicity Evidence: Provided pixel-wise statistical proof (over 18,580 frames) that SDR and HDR masters share a highly stable global monotonic luminance relationship ($R^2 > 0.99$ for most shots).
3. Residual Characterization: Identified and categorized sparse, structured local deviations into three physical types:
   • Type I: Self-luminous highlights (high energy, high structure).
   • Type II: Material-related structural regions (e.g., glass, metals).
   • Type III: Global baseline regions (low residual).
4. Operational Decision Map: Defined a method to separate regions where HDR recovers scene-referred data from regions where it performs creative, content-adaptive adjustments.

4. Key Results

Luminance Structure

• Global Stability: The mapping between SDR and HDR is highly monotonic. The coefficient of determination ($R^2$) exceeds 0.99 for over 85% of shots, with a film-wide mean of 0.9986.
• Geometric Consistency: Edge and texture structures are preserved with a gradient correlation coefficient ($\rho$) generally above 0.96.
• Residual Distribution:
  • Type III (Baseline): ~50% of pixels, contributing only 0.8% of residual energy.
  • Type I & II (Deviations): ~50% of pixels but contribute >99% of the residual energy.
  • Type I (Highlights): Occupies 18.3% of pixels but accounts for 95.4% of the total residual energy, primarily in self-luminous areas (e.g., sparks, spotlights) where HDR releases clipped highlights.

Color Structure

• Hue Stability: Hue shifts are minimal. The mean hue shift is 2.38°, with 95th percentile at 6.52°, indicating semantic color information is preserved.
• Saturation Redistribution:
  • Midtones (20–100 cd/m²): ~66.9% of pixels show saturation enhancement (limited release of color volume).
  • Shadows (<20 cd/m²): Predominantly desaturated (85.8% of pixels), likely due to perceptual adaptation to expanded contrast.
  • Highlights (>100 cd/m²): Saturation drops as luminance approaches the white point due to gamut constraints.

Decision Map Analysis (EXR Reference)

• Recovery vs. Adjustment: Using EXR as the ground truth anchor:
  • 82.4% of sampled regions are classified as EXR-closer recovery (HDR is perceptually closer to the original scene than SDR).
  • 17.6% are content-adaptive adjustments (HDR diverges from EXR to manage display constraints or creative intent).
• SNR Influence: In high-SNR scenes (e.g., Desert), HDR correlates strongly with EXR. In low-SNR scenes (e.g., Cave shadows), structural correlation drops significantly, suggesting literal physical recovery is unreliable in noisy shadows.

5. Significance and Implications

• Theoretical Shift: The study challenges the "physical inversion" assumption of traditional ITM. It proposes a "Restrained Restoration View": SDR-to-HDR adaptation is a structured remapping process that selectively releases information constrained by the display container while preserving the mastered narrative structure.
• Model Design: For learning-based SDR-to-HDR models, the findings suggest:
  • A strong global monotonic prior should be the baseline.
  • Model capacity should be concentrated on sparse, structured residuals (highlights, specific materials) rather than global inversion.
  • Low-SNR shadow regions should not be treated as recoverable physical data but rather as areas requiring perceptual reconstruction.
• Baseline for Research: Provides the first quantitative baseline for structure-aware SDR-to-HDR analysis using a common-source, professionally mastered dataset, offering a benchmark for future algorithmic development.

Conclusion: The paper demonstrates that in a controlled cinema workflow, HDR is not a random reconstruction but a highly structured, monotonic expansion of SDR, with specific, predictable deviations in highlights and materials. This understanding is crucial for developing next-generation tone-mapping algorithms that respect both physical scene data and creative mastering intent.