🔬 optics

Configurable Holography: Towards Display and Scene Adaptation

This paper introduces Configurable Holography, a learned framework that enables a single model to rapidly adapt to diverse display hardware and scene parameters—such as propagation distance, brightness, and pixel pitch—through explicit conditioning, thereby eliminating the need for retraining while achieving reconstruction quality comparable to existing methods with up to a 2x speed-up.

Original authors: Yicheng Zhan, Liang Shi, Wojciech Matusik, Qi Sun, Kaan Akşit

Published 2026-06-24

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Yicheng Zhan, Liang Shi, Wojciech Matusik, Qi Sun, Kaan Akşit

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to project a 3D hologram, like a floating image you can walk around. In the past, creating these holograms was like trying to bake a perfect cake for every single possible oven temperature, ingredient brand, and altitude. If you wanted to change the brightness, move the image closer or further away, or use a different projector, you had to start from scratch, retrain your "baker" (the computer model), and wait hours for a new recipe.

This paper introduces a new system called Configurable Holography. Think of it as a "Universal Hologram Chef" that can instantly adjust its recipe based on your instructions, without needing to relearn how to cook every time.

Here is how the paper breaks down this innovation, using simple analogies:

1. The Problem: The "One-Size-Fits-None" Approach

Currently, most hologram-making AI models are like a pair of shoes that only fit one specific foot size. If you want to change the display hardware (like the pixel size of the screen) or the scene (like how bright the image should be or how deep the 3D space is), the old model breaks. You have to throw it away and train a brand new one. This is slow, expensive, and prevents holograms from being flexible enough for real-world use.

2. The Solution: The "Swiss Army Knife" Model

The authors created a single AI model that acts like a Swiss Army knife. Instead of having a different tool for every job, this one tool has a dial you can turn to change its function instantly.

The Dials: You can adjust the "dials" for brightness, distance (how far the light travels), depth (how much 3D space is used), color wavelength, and pixel size.
The Magic: The model doesn't need to be retrained. You just tell it, "Make this hologram for a screen with 8-micron pixels, 10mm away, and very bright," and it adapts immediately.

3. The Secret Sauce: Learning from a "Master Chef" (Knowledge Distillation)

The "Master Chef" (called the Teacher Model) is incredibly smart and can handle all these adjustments perfectly, but it is slow and heavy, like a giant industrial oven. It takes over 10 seconds to bake a single hologram.

To make this practical for real-time use, the authors used a technique called Knowledge Distillation. Imagine a master chef teaching a fast, agile apprentice (the Student Model).

The apprentice watches the master work and learns the principles of the recipe, not just the specific steps.
The result? The apprentice is 16 times faster than the master (dropping from 651ms to just 39ms) and is twice as fast as previous state-of-the-art methods, while still producing a cake (hologram) that tastes almost exactly the same.

4. Seeing the Invisible: The "Depth Detective"

One of the hardest parts of making a 3D hologram from a flat 2D photo (like a standard JPEG) is figuring out how far away objects are. Usually, you need a special camera that measures depth.

The Innovation: The authors taught their model to be a "Depth Detective." They added a side-task where the model has to guess the depth of the image while it is making the hologram.
The Benefit: Even though the model isn't a professional depth-sensing expert, this "guessing game" helps it understand the 3D structure of the scene much better. This allows it to create accurate 3D holograms from regular 2D photos without needing special depth cameras.

5. The Results: Fast, Flexible, and Real

The team tested this system on three different physical holographic displays (different "ovens" with different pixel sizes).

Speed: They achieved a 2x speed-up compared to existing fast methods.
Quality: The images looked just as good as the old, slow methods that required retraining for every single setting.
Flexibility: They proved the model could handle continuous changes (e.g., moving the image smoothly from 2mm to 10mm away) without the image breaking or blurring.

Summary

In short, this paper presents a "smart, adaptable hologram generator." It solves the problem of holograms being too rigid and slow to use. By creating a single model that can be "tuned" like a radio station for different hardware and viewing conditions, and by training a tiny, super-fast version of that model, they have taken a major step toward making holographic displays that can actually adapt to how we want to see them, right now.

Technical Summary: Configurable Holography

Problem Statement

Rendering holograms for holographic displays is traditionally an iterative, computationally expensive process. While emerging learned Computer-Generated Holography (CGH) methods have accelerated rendering and improved reconstruction quality, they suffer from a critical lack of flexibility: existing models are trained on fixed display and scene parameters. Consequently, any change in hardware specifications (e.g., pixel pitch, wavelength) or scene requirements (e.g., propagation distance, volume depth, brightness) necessitates retraining a dedicated model. This rigidity creates a bottleneck for rapid adaptation to diverse user needs, such as adjusting focus ranges for different prescriptions or tuning prototypes during hardware development. Furthermore, most high-quality learned methods require RGB-D (depth) inputs, which are unavailable for standard 2D media like photos and videos.

Methodology

The authors propose Configurable Holography, a learned CGH framework where a single model adapts to diverse display-scene parameters through explicit conditioning, eliminating the need for retraining. The approach consists of three primary components:

Configurable Network Architecture:
- The system employs a teacher-student framework. The Teacher Model utilizes a U-Net architecture with an EfficientNet-B1 encoder and a Feature Pyramid Network (FPN) decoder.
- Explicit Conditioning: Display-scene parameters (peak brightness $s$ , propagation distance $Z$ , volume depth $VD$, wavelength $\lambda$ , and pixel pitch $dx$) are embedded using a novel combination of sinusoidal scalar encoding and a 1D Point Spread Function (PSF) derived from physical diffraction properties. These embeddings are injected into every decoder stage.
- Multi-Task Learning: The model performs three simultaneous tasks: predicting phase-only holograms, estimating light source powers for multi-color synthesis, and predicting monocular depth. This "hard-parameter sharing" allows the network to learn geometry relevant to 3D focus from RGB-only inputs.
RGB-Only Depth Estimation Strategy:
- To enable synthesis from standard 2D images, the model treats Monocular Depth Estimation (MDE) as an auxiliary task. While not competing with dedicated MDE models in accuracy, the depth head provides a supervisory signal that significantly improves the 3D focus cues in the generated holograms compared to methods that ignore depth.
Knowledge Distillation (KD) for Efficiency:
- To address the high inference cost of the teacher model (651 ms/frame), the authors apply Knowledge Distillation. A compact Student Model (2.19M parameters vs. 10.74M in the teacher) is trained to mimic the teacher's phase and depth predictions.
- The student uses a MobileNetV3 encoder and a lightweight decoder, achieving a 16.7× speedup over the teacher and a 1.9× speedup over existing state-of-the-art methods (Tensor V2) while preserving reconstruction quality.

Key Contributions

Configurable Holography Framework: The introduction of a single learned model capable of continuous adaptation across a range of display-scene parameters ( $s, Z, VD, \lambda, dx$ ) without retraining.
RGB-Only 3D Synthesis: A demonstration that joint learning of MDE and hologram synthesis enables accurate 3D hologram generation from RGB-only inputs, removing the dependency on depth maps.
Efficient Inference via Distillation: The successful application of KD to create a compact student model that achieves interactive inference rates (39 ms/frame) without sacrificing reconstruction fidelity.
Comprehensive Evaluation: Extensive quantitative and qualitative validation across three physical holographic display prototypes with varying pixel pitches (3.74, 6.4, and 8.0 µm) and numerous novel configurations.

Results

Reconstruction Quality: The configurable student model achieves on-par performance with existing learned CGH methods (e.g., Tensor V2, modified 3D NH) in terms of PSNR, SSIM, and perceptual metrics (LPIPS, FLIP). The student model maintains a PSNR difference of less than 0.1% compared to the teacher.
Speed: The student model offers up to a 2× speed-up compared to prior learned approaches (Tensor V2) under fp32 precision.
Generalization: The model successfully generalizes to unseen configurations within the trained ranges. For example, it maintains an average PSNR of ~~26 dB with low variance (~~1.1 dB) across continuous ranges of propagation distance and pixel pitch.
Hardware Validation: Captured results from three different display prototypes confirm that the method preserves correct focus/defocus cues and color accuracy across different hardware configurations and brightness levels.

Significance and Claims

The authors position this work as an initial step toward flexible, general-purpose learned holography systems. They emphasize that the primary contribution is not the specific network architecture (which relies on established techniques like U-Net, MTL, and KD), but the formulation of configurability as a concrete objective for the field.

The paper claims to provide the first empirical study quantifying the trade-offs between speed, parameter range, input requirements, and image quality across thousands of novel configurations. It highlights that while achieving configurability is non-trivial due to the complex, non-uniform conditioning difficulty of different parameters (e.g., long-range propagation is harder than brightness adjustment), a single model can effectively approximate a global optimal solution across diverse settings.

The authors remain modest, acknowledging that their current implementation is a prototype. They note limitations such as performance degradation when extrapolating beyond trained parameter ranges and the inherent difficulty of stabilizing quality across wide conditioning ranges. They conclude that future work must address robust generalization to novel configurations without dense retraining and extend configurability to emerging display architectures and non-ideal hardware conditions.