BAG: Budget-Aware Gating for Diffusion Caching
The paper introduces BAG (Budget-Aware Gating), a novel caching policy for Diffusion Transformers that employs a lightweight gating network trained via schedule distillation to dynamically balance global budget constraints with instance-adaptive feature reuse, thereby outperforming existing methods in accelerating diffusion models like FLUX.1-dev and Wan2.1.
Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
Imagine you are trying to paint a masterpiece, but instead of a brush, you are using a super-smart robot that has to take 50 tiny steps to finish the picture. Each step involves the robot thinking hard, looking at the canvas, and deciding what color to add next. This is how modern AI creates images and videos: it starts with a noisy blur and slowly "denoises" it into a clear picture. The problem is that taking 50 steps is slow and expensive, like hiring a robot to walk across the country just to paint a single flower.
To speed things up, scientists have tried two main tricks. The first is like a strict teacher who says, "You must take a break every 5 steps," regardless of what the robot is doing. This is fast, but sometimes the robot needs to think hard right then, and the break ruins the picture. The second trick is like a student who says, "I'll only stop if the picture looks really different," without checking how much time is left. This adapts to the picture, but the student might run out of time before finishing or waste energy on easy parts. Both methods have a flaw: they can't see the whole picture of time and effort at once.
This is where the Westlake AGI Lab's new paper, "BAG: Budget-Aware Gating for Diffusion Caching," comes in. The researchers built a tiny, super-smart "traffic controller" for the robot. Instead of following a rigid schedule or guessing based on a single clue, this controller looks at two things at every step: how much "energy" (or computer steps) is left in the tank, and how much the picture is actually changing right now. It learns from a massive amount of offline practice to know exactly when to take a shortcut (reusing a previous calculation) and when to do the full work.
The paper finds that this new "traffic controller" is much better than the old methods. When tested on powerful image and video generators, the BAG system consistently produced clearer, more accurate pictures and videos than the best existing shortcuts, even when forced to use the exact same amount of computer power. It managed to be about 5 times faster than the standard method without losing quality, and it worked perfectly whether the user wanted a fast, rough sketch or a slower, detailed masterpiece. The researchers showed that by teaching the system to balance the "budget" of time with the "mood" of the picture, they could get the best of both worlds: speed without the mess.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.