🔬 optics

Research and Development of High-Efficiency Compressed Transmission Technologies for Holographic Communication in Beyond 5G Networks

This report outlines the research outcomes from Hokkaido University's NICT-funded project on developing high-efficiency compressed transmission technologies for holographic communication in Beyond 5G networks, presenting specialized findings in an accessible format for a broad audience.

Original authors: Yuji Sakamoto, Takahiro Yamanoi, Seok Kang

Published 2026-07-07

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Yuji Sakamoto, Takahiro Yamanoi, Seok Kang

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to send a perfect, three-dimensional hologram of a rabbit to a friend. In the past, this was like trying to mail a library's worth of books just to show a single picture. The data is so huge that it would clog the internet, and the computers needed to create it would take hours to finish a single frame.

This report from Hokkaido University describes how they built a "magic pipeline" to solve these problems, aiming to make holographic TV (Holo-TV) a reality for the next generation of internet (Beyond 5G).

Here is how they did it, broken down into simple concepts:

1. The "Universal Translator" (Common Data Format)

The Problem: Imagine you are sending a movie to a friend. If your friend has a giant TV, a tablet, or a pair of glasses, you usually have to re-edit the movie specifically for their device. If you have 1,000 friends with 1,000 different devices, you'd have to make 1,000 different versions. That's impossible.

The Solution: The researchers created a "Universal Translator" for light. Instead of sending a movie tailored to a specific screen, they send a single, raw "light recipe" (called Object Light) from the broadcast station.

How it works: Think of this recipe as a set of instructions for how light should look at your eye. When the signal reaches your device (whether it's a headset or a table-top projector), your device acts like a translator. It takes that universal recipe and instantly converts it into the specific format its screen needs.
The Result: The broadcaster only sends one signal, but every user gets a perfect 3D image on their specific device.

2. The "Super-Fast Kitchen" (High-Speed Computing)

The Problem: Creating these holograms is like baking a cake where you have to calculate the exact path of every single crumb of light. Doing this for a video in real-time used to take hours.

The Solution: They built a massive "kitchen" using a cluster of 18 powerful graphics cards (GPUs), which is like having 90,000 chefs working together.

The Trick: They organized the work like an assembly line. While one group of chefs preps the ingredients, another cooks, and another plates the food.
The Result: They managed to cook a full second of holographic video in less than a second (about 25 frames per second). This means the "rabbit" on the screen can jump, bounce, and move in real-time without lag.

3. The "Magic Mirror" (Realistic Reflections)

The Problem: In the real world, light bounces off curved surfaces like doorknobs or car bumpers. Old computer methods struggled to calculate these bounces, making holograms look flat or fake.

The Solution: They developed a new math trick to trace light rays bouncing off curved mirrors.

The Analogy: Imagine trying to predict where a ball will bounce if you throw it at a curved wall. Instead of guessing, they used a method called "Bézier clipping" to map the curve perfectly.
The Result: They created holograms where you can see a rabbit reflected in a mirror behind it, and even see a second reflection of that reflection. It looks incredibly real, with correct depth and shadows.

4. The "Smart Glasses" (Head-Mounted Displays)

The Problem: If you turn your head, the 3D image needs to update instantly, or it will look like a flat sticker stuck to your forehead.

The Solution: They built a lightweight headset (Holo-HMD) that tracks your head movements.

The Trick: Instead of recalculating the whole image when you turn your head, the device only does a tiny "tweak" to the light's angle.
The Result: You can look left, right, up, or down, and the 3D rabbit stays in place, just like a real object. It updates fast enough (over 30 times a second) that your brain doesn't notice the computer working.

5. The "Lazy Eye" Trick (Foveated Rendering)

The Problem: Holograms require massive amounts of data. Sending a 4K hologram is like trying to stream a movie in 8K resolution—it's too heavy for current networks.

The Solution: They used a trick based on how human eyes work. Your eyes only see high detail in the very center (where you are looking); the edges of your vision are blurry.

The Analogy: Imagine painting a portrait. You paint the eyes with extreme detail, but you paint the background with broad, loose brushstrokes.
The Result: The computer only calculates the high-detail "center" of the image. The blurry "edges" are calculated with less effort. This shrinks the data size by more than three times and makes the computer work eight times faster, without you noticing the difference.

6. The "Floating Rabbit" (Real-World Testing)

The Proof: They didn't just do math; they tested it. They took real 3D scans of a person and a rabbit, sent the data through their system, and displayed it on the headset.

The Result: When the researchers moved their heads, the rabbit and the person moved with them, showing different angles and hiding behind objects just like real life.

Summary

This project successfully built the "engine" for a future where you can watch 3D holographic TV. They solved the three biggest hurdles:

Data Size: By using a universal format and "lazy eye" compression.
Speed: By using a super-computer cluster and smart math.
Realism: By calculating complex reflections and matching human vision.

The report concludes that while the individual parts work perfectly, the next step is to connect them all into one complete, end-to-end system to show the world what the future of 3D communication looks like.

Technical Summary: Research and Development of High-Efficiency Compressed Transmission Technologies for Holographic Communication in Beyond 5G Networks

1. Problem Statement

The practical realization of holographic television (Holo-TV) and immersive communication has been historically hindered by two primary bottlenecks: the enormous volume of hologram data required for high-fidelity reconstruction and the limited resolution and computational capabilities of display devices. Traditional approaches often require transmitting hologram data tailored to specific user devices and viewpoints, resulting in unsustainable computational loads at the broadcasting station and excessive communication bandwidth requirements. Furthermore, existing Computer-Generated Holography (CGH) algorithms struggle with realistic rendering of complex scenes (e.g., curved reflections, hidden surfaces) and real-time performance, particularly when supporting six degrees of freedom (6DoF) and diverse display architectures (e.g., Head-Mounted Displays, benchtop projectors).

2. Methodology

The research group at Hokkaido University proposed a system architecture termed Holometric Video Streaming (HVS). This framework decouples the broadcasting process from the user's specific device and viewpoint. The core methodology involves:

Common Data Format: Instead of transmitting device-specific holograms, the system transmits "common data" (specifically, object light at the pupil position, $O_{eye}$ ) that is independent of the user's hardware. The receiving device then performs a back-propagation calculation to convert this common data into hologram data ( $O_{SLM}$ ) suitable for its specific optical system.
Realistic Rendering via Ray Tracing: To achieve photorealism, the group developed ray-tracing algorithms capable of handling hidden surface removal and reflections on curved surfaces (using Bézier surfaces and Bézier clipping).
High-Speed Computation:
- Server Side: A GPU cluster (18 NVIDIA RTX 4090 GPUs) was utilized to parallelize CGH computation, employing pipeline processing and spatial division of the hologram plane.
- Cylindrical Recording: To support free-viewpoint viewing, a cylindrical recording surface was introduced. An algorithm was developed to efficiently compute object light on this cylinder by back-propagating from a planar surface, achieving a speedup of over 10,000 times compared to direct calculation.
- Client Side: Lightweight phase compensation algorithms were developed for user devices (HMDs) to update holograms in real-time based on head rotation, independent of the number of point light sources.
Human Visual System (HVS) Exploitation: To address compression and computation speed, foveated rendering was applied. This technique renders the central gaze region (fovea) at high resolution while reducing the resolution of peripheral regions using "sub-holograms."
Hardware Development: The project involved the design and construction of various display prototypes, including a Holographic Head-Mounted Display (Holo-HMD), a Holographic Head-Mounted Projector (Holo-HMP), portable FPGA-based data generation devices, and benchtop wide-field-of-view displays.

3. Key Contributions and Results

A. System Architecture and Data Formats

HVS Framework: Successfully demonstrated a broadcast-type architecture where common data is transmitted, and device-specific holograms are generated locally. This significantly reduces the computational load on the broadcaster and the bandwidth required for transmission.
Object Light as Common Data: Validated that transmitting the optical wave at the pupil position ( $O_{eye}$ ) allows for the reconstruction of identical images across different optical systems (e.g., with and without convex lenses) after appropriate inverse transformation.

B. Rendering and Computation Performance

Curved Reflection Rendering: Developed an algorithm for generating reflections on curved surfaces (e.g., metallic doorknobs, curved mirrors). By subdividing Bézier surfaces into planar segments and applying Fermat's principle, computation speed was accelerated by over 30 times, and further by 100 times with optical path optimization, enabling generation times on the order of seconds.
GPU Cluster Acceleration: A 4-server cluster with 18 GPUs achieved a theoretical peak of 2 PFLOPS. This enabled the generation of 2K resolution holographic video at 25.8 frames per second (FPS) for complex physical simulations (e.g., a rabbit on a seesaw), achieving quasi-real-time performance.
Cylindrical Object Light: The proposed back-propagation method for cylindrical recording surfaces achieved a 10,000-fold speedup over conventional point-light source methods.

C. Display Devices and User Experience

Holo-HMD: A compact (125 × 130 × 40 mm), full-color HMD with a 16.4° field of view (FOV) was developed. It supports 6DoF and updates holograms at >30 FPS in response to head motion.
Holo-HMP: A head-mounted projector system was realized that projects holographic images onto a concave mirror, expanding the FOV without obstructing the user's view of the real world. It includes real-time phase compensation for head roll.
Benchtop Displays: Devices with wide FOVs (12.8°) and wide viewing zones (32°) were demonstrated. These systems utilize eye-tracking to electronically switch reconstruction light, allowing multiple users or a single user to view the image from various angles without physical adjustment.
Speckle Noise Reduction: Two methods were tested:
1. Temporal Coherence Reduction: Using yellow phosphor excitation and frequency filters reduced speckle noise by ~30% while maintaining brightness.
2. Spatial Coherence Reduction: Vibrating multimode optical fibers reduced noise by 43%, though with some trade-off in image resolution.

D. Compression and Efficiency

Foveated Rendering: By applying sub-holograms to peripheral vision, the study achieved a speedup of >8 times in computation compared to full-resolution rendering.
Compression Ratio: The reduction in data volume via sub-holograms allowed for a compression ratio improvement of more than 3 times compared to conventional lossless compression methods.
Real-World Data: The system successfully displayed 3D data captured from real-world objects (using polygon and point cloud data) on the Holo-HMD, demonstrating 6DoF free-viewpoint capabilities with realistic occlusion and motion parallax.

4. Significance and Claims

The authors claim that this research establishes the fundamental elemental technologies required for broadcast-type holographic communication in Beyond 5G networks. The significance of the work lies in:

Scalability: The HVS architecture solves the scalability issue of broadcasting to diverse devices by shifting the computational burden of device-specific rendering to the user side while maintaining a single common data stream.
Feasibility of Real-Time Holo-TV: By combining GPU cluster acceleration, optimized rendering algorithms, and HVS-based compression, the project demonstrates that real-time, high-fidelity holographic video transmission is technically feasible, meeting the target of 1.5 Gbps or lower (an eightfold efficiency improvement over conventional methods).
Practical Device Integration: The development of portable, lightweight, and wide-viewing-zone display devices (HMDs, HMPs, benchtop units) bridges the gap between theoretical holography and practical consumer applications.
Foundation for Future Systems: While the study focused on individual elemental technologies, it provides a validated roadmap for integrating these components into an end-to-end system for immersive 3D communication, education, and healthcare applications.

The report concludes that while further work is needed for full end-to-end system integration and further acceleration, the proposed HVS framework and the demonstrated technologies effectively address the critical barriers to realizing holographic communication in the Beyond 5G era.