Predicting total time to compress a video corpus using online inference systems

Imagine you are running a massive digital library where people constantly drop off thousands of video files to be shrunk down for streaming. This process, called "compression," is like packing a suitcase: you have to fold and squeeze the video data so it takes up less space, but the harder the video is to fold (lots of fast motion or complex textures), the longer it takes. In the world of cloud computing, knowing exactly how long this packing job will take is a huge deal. It helps companies figure out how much power they need to burn and, more importantly, how much to charge their customers. Right now, most systems act like a blindfolded chef; they only tell you the price of the meal after you've already eaten it. This makes budgeting a nightmare. Scientists have tried to predict the time for each individual video clip, but that's like trying to guess the total weight of a whole truckload of bricks by weighing just one brick at a time. It's slow, and the errors add up. The big question is: can we predict the total time for the entire pile of videos before we even finish the job, and can we get smarter about it as we go?

This paper, written by a team from Trinity College Dublin, tackles that exact problem. Instead of guessing the time for every single video clip one by one, they built a new kind of "smart calculator" that predicts the total time needed to compress a whole collection (or "corpus") of videos. They found that looking at the big picture is much better than looking at the small pieces. In fact, their method of predicting the total time for the whole group was more than twice as accurate as the old method of guessing per clip.

Here is how their system works, using a fun analogy. Imagine you are a foreman at a construction site with 600 different rooms to paint. Some rooms are tiny and plain; others are huge and covered in intricate murals.

• The Old Way (Per-Clip Prediction): You try to guess how long it will take to paint every single room before you start. You might guess the mural room takes 10 hours and the closet takes 1 minute. But if you get even one guess wrong, your total estimate for the whole job is off.
• The New Way (Corpus Prediction): You start painting. After you've finished just a few rooms (say, 2% of the job), you stop and look at what you've done. You realize, "Hey, the rooms I've painted so far are actually taking 15 minutes each on average." You then use that real-world data to guess how long the rest of the 588 rooms will take.

The authors tested this idea with two different "painting tools" (video codecs called x264 and x265) and a huge dataset of 600 high-quality 4K video clips. They didn't just guess; they built a system that learns as it goes. They call this "online inference." It's like a GPS that updates your route and arrival time every time you pass a new landmark, rather than just giving you a static map at the start.

They tested several strategies:

1. The "Progress Bar" Guess: This is the simplest method. It just assumes the rest of the videos will take the same average time as the ones you've already finished. It's okay, but not great.
2. The "Grouping" Guess: This method sorts the videos into groups (clusters) based on how complex they look (like sorting rooms by whether they have murals or plain walls). It predicts the time for the remaining rooms in each group separately. This is better.
3. The "Super-Brain" Guess (Machine Learning): This uses a smart algorithm called XGBoost. It looks at the details of the videos you've already processed and learns a complex pattern to predict the rest.

The results were surprising and impressive. The authors found that you don't need to use the "Super-Brain" for the whole job. In fact, the best strategy is a mix-and-match approach:

• Before you start: Use a general "Super-Brain" model to get a rough idea. For the x264 tool, this guess was off by about 13.5% of the total time.
• After 2% is done: Switch to the "Super-Brain" that learns on the fly (online). This drops the error to about 8.75%.
• After 6% is done: Switch to the simpler "Grouping" method. Surprisingly, this simple method became even more accurate, dropping the error to just under 5% (4.89% for x264 and 5.11% for x265).

The paper explicitly argues against the idea that you need a single, complex model trained on everything to get good results. They showed that general models (trained on data from other sources) actually performed worse once the actual job started. They also proved that you don't need to wait until the job is 50% or 90% done to get a good estimate; you can get very accurate predictions (less than 5% error) after processing just a tiny fraction of the videos.

The team measured this on a powerful computer with 64 cores, processing 600 clips that were either 2 or 4 seconds long. They found that the "brain power" needed to run these predictions was tiny—adding less than 0.2% extra time to the whole process. This means the system is fast enough to be used in real-time without slowing down the actual video compression.

In short, the paper suggests that for predicting how long a massive video job will take, it's better to be a "learning foreman" than a "crystal ball." By watching a small sample of the work and adjusting your guess as you go, you can predict the total cost and time with much higher accuracy than trying to guess every single step in advance. This helps cloud companies save money and lets customers know exactly what they are paying for before the work even begins.