Plans You Can Check: Verifier-Grounded Learning of an Open-Weight Planner for Executable Video-Editing
This paper introduces RefineCut, an open-weight planner for executable video editing that leverages a deterministic verifier to distill multi-teacher repairs and refine preferences, enabling an 8B model to outperform frontier teachers on a new benchmark without requiring teacher calls during inference.
Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
In the world of artificial intelligence, a common distinction separates the act of creating new images from the act of organizing existing ones. While many systems have learned to paint pixels from scratch, turning a simple idea into a brand-new picture, the challenge of editing a video is different. Real-world video editing is less about generating new light and more about making a series of strict decisions. An editor must look at a pool of existing footage, decide which clips to keep, determine how long each should play, figure out where to place them, and ensure the cuts match the rhythm of a soundtrack. This process is a puzzle of constraints: the final video must be a specific length, include certain scenes, exclude others, and stay perfectly synchronized with music. For a computer to do this, it cannot simply guess; it must follow a plan that can be checked against a list of rules.
A team of researchers has tackled this problem by treating video editing not as a creative guessing game, but as a verifiable planning task. They built a system called RefineCut, which teaches a compact, open-source computer model to create these editing plans. Unlike previous approaches that rely on massive, closed-source models to guess the right answer, RefineCut uses a two-step learning process. First, it takes the messy, often conflicting suggestions from several powerful AI teachers and runs them through a strict, automated checker. This checker, which acts like a digital editor, tests every proposed change against a clear list of requirements, such as "the video must be 30 seconds long" or "this specific clip must be included." Only the suggestions that pass this test are kept as examples for the student model to learn from. This ensures the model learns from verified successes rather than copying the mistakes or guesses of its teachers.
The researchers then introduced a second stage of training where the model learns to improve its own work. Instead of waiting for a human or a teacher to grade its edits, the model generates several possible fixes for a problem, and the automated checker scores them based on how well they meet the rules. The model is then trained to prefer the highest-scoring fixes. This cycle allows the system to refine its own planning abilities until it can operate entirely on its own, without needing to ask a larger, more expensive AI for help. The result is a system that can take a brief description of a video, a library of clips, and a set of rules, and produce a detailed, executable plan to assemble the video.
To test this approach, the team created a new benchmark called RefineCut-Bench, which contains thousands of editing tasks, real video clips, music tracks, and explicit lists of constraints. They found that when the model learned directly from the raw suggestions of powerful AI teachers, it performed poorly, achieving a score of 0.620 on their specific testing scale. However, once the model was trained using the verified, checker-approved suggestions, its performance jumped significantly to 0.858. After the second stage of self-improvement, where the model learned to choose the best of its own repairs, the score rose further to 0.924. This final version of the model, which is relatively small and efficient, performed as well as or better than the much larger, more complex AI systems it was originally trained to mimic.
The study explicitly rules out the idea that simply copying the output of powerful AI models is enough to solve this problem. The researchers showed that when the model tried to imitate the teachers directly, it inherited their errors and inconsistencies. The key to success was the intermediate step of verification. By replaying every possible editing decision through the automated checker before using it as a teaching tool, the system filtered out the noise and kept only the reliable paths. This method proved robust across different types of AI models, not just the one they started with, suggesting that the ability to verify and check work is a more powerful teacher than the sheer size of the model itself.
The researchers also tested whether these planning gains translated into actual visual quality. They had human judges compare blind, left–right randomized A/B previews (storyboard renders) produced by the new system against those from the older methods. The judges consistently preferred the videos made by the RefineCut system, confirming that the improved planning scores matched a better visual result. The system was able to handle complex requests, such as replacing a specific scene, adjusting the pacing to match a beat, or ensuring the total duration was exact, all while avoiding the common errors of missing clips or mismatched lengths that plagued the unverified models.
This work demonstrates that for tasks involving strict rules and structured decisions, a smaller, open model can outperform massive, closed systems if it is trained with a reliable way to check its own work. The researchers did not claim to have solved every aspect of video creation, such as judging the artistic taste of a cut or understanding complex emotional stories, which remain difficult for machines. However, they successfully showed that the mechanical side of editing—selecting, ordering, and timing clips to meet a set of requirements—can be mastered by a system that learns from a cycle of planning, checking, and self-correction. The code and the dataset used for these experiments are now publicly available, allowing other researchers to build upon this verified approach to machine planning.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.