Scalable On-the-fly Transcoding for Adaptive Streaming of Dynamic Point Clouds

This paper presents and evaluates a scalable dynamic point cloud streaming system that leverages on-the-fly transcoding, demonstrating how caching and speculative transcoding significantly reduce server loads and improve user Quality of Experience to support a higher number of simultaneous clients.

The Gist

Imagine you are running a massive, high-tech 3D movie theater where the audience can walk around inside the movie, look at things from any angle, and even touch the characters. This is what "Dynamic Point Clouds" are: 3D scenes made of millions of tiny dots (points) that create a realistic, immersive world.

The problem? These 3D movies are huge. Storing every single version of the movie (from low quality for slow phones to ultra-high quality for VR headsets) would fill up your hard drives instantly and cost a fortune.

This paper proposes a clever solution: The "On-the-Fly" Chef.

The Problem: The "All-Prepared" vs. "Cook-It-Now" Dilemma

The Old Way (Pre-encoding):
Imagine a restaurant that prepares every single dish on the menu in advance and keeps them warm in the kitchen.

• Pros: When you order, the food is ready instantly.
• Cons: If nobody orders the "Spicy Tofu," you wasted money and space keeping it warm. If you have 1,000 different menu items, your kitchen is a disaster zone of wasted food.

The New Way (On-the-Fly Transcoding):
Imagine a restaurant that only keeps the Master Ingredient (the highest quality, most detailed version of the dish) in the kitchen. When you order, the chef instantly cooks it down to the size and spice level you requested.

• Pros: You save massive amounts of storage space. You only keep one "Master" file.
• Cons: Cooking takes time. If 50 people order at once, the chef gets overwhelmed, the kitchen backs up, and your food arrives late. In a video stream, "late food" means the video freezes (buffers) or drops in quality.

The Solution: The Smart Kitchen Team

The authors built a system to make this "Cook-It-Now" approach work for thousands of people at once. They tested three "kitchen hacks" to stop the chef from getting overwhelmed:

1. The "Leftover" Shelf (Caching)

• The Metaphor: If Chef A just made a "Medium Spicy Burger" for Customer 1, and Customer 2 immediately orders the exact same thing, the chef doesn't cook it again. They just grab it from a small shelf right next to the stove.
• The Result: This saves a huge amount of time because most people tend to watch the same parts of a video or stick to the same quality level.

2. The "Next Order" Guess (Speculative/ Predictive Transcoding)

• The Metaphor: The chef knows that after you eat a burger, you usually order fries. So, while you are eating, the chef guesses you'll want fries and starts cooking them before you even ask.
• The Result: When you actually ask for fries, they are ready instantly.
• The Catch: If you decide to leave instead of ordering fries, the chef wasted energy cooking them. If the restaurant is too busy, this "guessing" actually slows things down because the chef is too busy guessing to cook the actual orders.

3. The "Emergency Snack" (Fallback Pre-encoding)

• The Metaphor: The chef keeps a few "Emergency Snacks" (low-quality, fast-to-serve items) pre-made on the counter. If the kitchen is totally swamped and the chef is stuck cooking a complex steak, the chef can immediately hand you a snack to keep you from getting hungry (buffering).
• The Result: This is the most effective trick for busy times. It ensures that even when the system is stressed, the video doesn't freeze completely; it just drops to a lower quality temporarily.

What They Found

The researchers tested this system with up to 40 "customers" (viewers) at the same time. Here is what happened:

• Just Cooking (No Hacks): If you just rely on the chef cooking everything from scratch, the system crashes quickly. Viewers experience constant freezing (stalls) because the chef can't keep up.
• Cooking + Leftover Shelf: This helps a lot. It smooths out the experience for moderate crowds.
• Cooking + Guessing: This works great for small crowds, but if too many people show up, the chef gets distracted by guessing and the system gets slower.
• Cooking + Leftovers + Emergency Snacks: This is the winner. By keeping a few low-quality "snacks" ready and using the leftover shelf, the system can handle a massive crowd without freezing, even if the chef is working at full speed.

The Bottom Line

This paper proves that you don't need to store terabytes of data to stream high-quality 3D movies to millions of people. Instead, you can store one master version and use a smart, fast kitchen (transcoding system) with a few clever tricks (caching and emergency backups) to serve everyone instantly.

It's like moving from a warehouse full of pre-made meals to a super-efficient, high-tech kitchen that can whip up a gourmet meal for 1,000 people in seconds, as long as you have a few snacks ready for the rush hour.

Technical Summary

Here is a detailed technical summary of the paper "Scalable On-the-fly Transcoding for Adaptive Streaming of Dynamic Point Clouds."

1. Problem Statement

The delivery of immersive Augmented and Virtual Reality (AR/VR) content, specifically dynamic point clouds, faces significant challenges regarding storage and scalability.

• Storage Inefficiency: Traditional Adaptive Bitrate (ABR) streaming requires pre-encoding and storing multiple bitrate representations (a "bitrate ladder") for every content segment. Since most content is viewed infrequently (following a power-law distribution), storing all representations for every segment results in massive, often wasted, storage overhead.
• Transcoding Latency: While on-the-fly transcoding (generating lower-bitrate versions from a high-bitrate source upon request) solves the storage issue, it introduces significant computational load and latency.
• Scalability Gap: For Video-based Point Cloud Compression (V-PCC), re-encoding the underlying video bitstreams is efficient, but it remains unclear if such a system can scale to support many simultaneous clients without degrading the Quality of Experience (QoE) due to playback stalls caused by high server response times.

2. Methodology

The authors propose and evaluate a scalable streaming system architecture that combines on-the-fly transcoding with specific server-side optimization strategies.

System Architecture

• Frontend: A media server (NGINX/FastAPI) handles client requests.
• Backend: A pool of transcoding workers (GPU-enabled nodes) using the RABBIT transcoder to re-encode V-PCC bitstreams.
• Workflow: When a client requests a segment:
  1. The server checks local storage (for pre-encoded high/low rates).
  2. If not found, it checks a cache for previously transcoded segments.
  3. If not cached, it queues a transcoding job.
  4. Speculative Execution: Upon serving a request, the system proactively queues a job for the next segment (assuming the client will stay on the same quality).

Experimental Setup

• Testbed: 8 bare-metal nodes (imec Virtual Wall).
• Content: 4 dynamic point cloud sequences from the 8iVFBv2 dataset (extended to 80s), encoded with V-PCC (R5 configuration).
• Clients: Simulated DASH players (iStream) with configurable buffer levels (target 12s, panic 2s) and emulated network traces.
• Variables:
  • Number of clients (4 to 40).
  • Number of transcoding workers (1 vs. 2 GPU nodes).
  • Segment length (2s vs. 4s).

Evaluated Configurations

The study compares several strategies against a Baseline (B) where all representations are pre-encoded:

1. T (Transcoding): Pure on-the-fly transcoding (no cache, no pre-encoding).
2. T+C: Adds an LRU Cache for transcoded segments.
3. T+C+P: Adds Predictive Transcoding (speculative jobs for the next segment).
4. T+C+F: Adds Fallback Pre-encoding (storing the lowest bitrate representation on the server).
5. T+C+F+P: Combines all optimizations.

3. Key Contributions

1. System Evaluation: The first empirical evaluation of the scalability limits of on-the-fly transcoding for dynamic point clouds under varying client loads.
2. Optimization Strategies: Introduction and validation of three specific techniques to mitigate latency:
   • Caching: Reusing recently transcoded segments.
   • Speculative Transcoding: Pre-emptively processing the next segment.
   • Fallback Pre-encoding: Storing the lowest bitrate to ensure immediate service during buffer starvation.
3. QoE Analysis: A detailed correlation between server-side response latency and client-side playback stalls, demonstrating how server delays directly impact ABR behavior.

4. Results

The experiments reveal critical insights into the trade-offs between storage, computation, and QoE:

• Latency Impact: Pure transcoding (T) introduces high response latency. Even with sufficient resources, ~40% of requests are instant (served from storage), but the remaining 60% suffer from queuing delays, leading to frequent stall events (buffer under-runs).
• Effectiveness of Caching (+C): Caching significantly reduces the number of transcoding jobs. As the number of clients increases, caching becomes the most effective single metric for reducing stalls, as it allows the system to serve repeated requests instantly.
• Limitations of Speculation (+P): Predictive transcoding helps with small client counts (4 clients) by reducing latency for the next segment. However, at scale (24+ clients), it increases the load on the job queue. If the speculative segments are never requested, the system wastes resources, leading to faster queue exhaustion and more stalls.
• Superiority of Fallback Pre-encoding (+F): Storing the lowest bitrate representation (T+C+F) proved to be the most robust strategy for scaling.
  • Clients often drop to the lowest bitrate when their buffer is low.
  • Serving this immediately prevents stalls without needing transcoding.
  • This configuration allowed the system to scale to 40 clients with stall rates comparable to the Baseline (pre-encoded all), whereas pure transcoding failed.
• Bitrate Distribution: With the T+C+F+P configuration, the system successfully mimicked the bitrate distribution of the full Baseline (serving a mix of R1–R4) even under load, proving that on-the-fly transcoding can replace full pre-encoding without sacrificing visual quality diversity.

5. Significance

This work demonstrates that on-the-fly transcoding is a viable solution for scalable dynamic point cloud streaming, provided that specific server-side mitigations are employed.

• Storage Reduction: It validates that providers do not need to store massive amounts of pre-encoded data, significantly reducing storage costs for long-tail content.
• Scalability Path: It identifies that a "transcoder-only" approach is insufficient for large-scale deployment. Instead, a hybrid approach (Caching + Fallback Pre-encoding) is required to maintain QoE.
• Future Directions: The paper suggests that exposing transcoding time estimates to the client (via the MPD) could allow ABR algorithms to make smarter decisions, potentially trading visual quality for smoother playback during high-load periods.

In conclusion, the paper provides a roadmap for deploying immersive point cloud streaming services that are both storage-efficient and capable of handling high concurrency without compromising user experience.