💬 NLP

Crayotter: Learning Long-Horizon Video Editing Agents via Group-Relative Preference Backpropagation

The paper introduces Crayotter, a 9B video editing agent trained via Group-Relative Preference Backpropagation (GRPB), which converts subjective long-horizon editing feedback into ordinal comparisons to effectively redistribute credit across semantic segments, thereby outperforming proprietary systems on benchmarks like AgenticVBench.

Original authors: Lecheng Yan, Jianze Lin, Yichong Zhang, Ben Pan, Wenxi Li, Chenyang Lyu, Liting Zhou, Cathal Gurrin

Published 2026-08-05

📖 7 min read🧠 Deep dive

CC BY 4.0

Original authors: Lecheng Yan, Jianze Lin, Yichong Zhang, Ben Pan, Wenxi Li, Chenyang Lyu, Liting Zhou, Cathal Gurrin

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are teaching a robot to be a movie director. You give it a pile of raw video clips and a simple instruction: "Make a cool sports highlight reel." The robot has to do a lot of work: it has to find the right clips, cut them, arrange them in a timeline, add music, and export the final file. This is a long process with many steps. The problem is that you only get to see the final movie at the very end. If the movie is boring, you can't easily tell the robot which specific step went wrong. Did it pick the wrong clips? Did it cut them too short? Or did it just arrange them in a bad order? In the world of artificial intelligence, this is called the "long-horizon" problem. It's like trying to teach someone to bake a cake by only telling them "it tastes bad" after they've already eaten the whole thing. You need a way to give credit (or blame) to the specific steps that led to the result, even if the result is subjective—meaning different people might like different versions of the same video.

This paper introduces a new way to teach these AI editors, called Crayotter. The researchers realized that instead of trying to give the robot a single "score" for the whole video (which is hard because "good" is in the eye of the beholder), they should just ask: "Between two different versions of the same video, which one is better?" By comparing videos made from the exact same starting materials, the AI can learn a clear ranking. But the tricky part is still figuring out where in the long process the AI made the good or bad choices. The authors propose a method called Group-Relative Preference Backpropagation (GRPB). Think of it as a smart referee that doesn't just shout "Good job!" or "Bad job!" at the end of the game. Instead, it looks at the whole team's performance, figures out who contributed to the win, and gives credit to the specific players (or in this case, the specific editing steps) that mattered most, while making sure no single step gets too much credit or too much blame.

The Problem: The "Black Box" of Video Editing

When an AI tries to edit a video, it goes through a long chain of decisions. It analyzes the footage, picks clips, builds a timeline, and renders the final product. The problem is that the quality of the final video—how well the story flows, if the pacing feels right, or if the style matches the request—is only visible after all those decisions are made.

If you just give the AI a single number at the end (like "This video is a 7 out of 10"), it's confusing. Is the 7 because the AI picked bad clips? Or because the music was too loud? Or because the task itself was just really hard? Different editing tasks have different rules for what makes a "good" video, so comparing scores across different tasks is like comparing apples to oranges.

The paper argues that we shouldn't try to calculate a perfect global score. Instead, we should focus on comparisons. If the AI makes three different versions of the same sports reel, we can easily say, "Version C is better than Version A, and Version A is better than Version B." This turns a vague, subjective feeling into a clear, logical order.

The Solution: GRPB and the "Lagged" Credit System

The authors introduce Group-Relative Preference Backpropagation (GRPB). Here is how it works, using a playful analogy:

Imagine a cooking competition where three teams (Team A, Team B, and Team C) are given the exact same ingredients and asked to make a stew. A judge tastes all three and ranks them: Team C is the winner, Team A is second, and Team B is the loser.

In old methods, the judge might just give the winning team a big trophy and the losing team a participation ribbon. But that doesn't tell the chefs what they did right or wrong. Did Team C win because they chopped the onions better? Or because they added the spices at the right time?

GRPB is like a super-smart coach who breaks down the game.

The Group: It only compares teams that used the exact same ingredients (the same request and source clips).
The Zero-Sum Game: It treats the ranking like a zero-sum game. If Team C wins, it gains points, and the others lose points. The total points in the room always add up to zero.
The Lagged Allocator: This is the clever part. The coach doesn't just look at the final stew and guess. It uses a "lagged" system. It looks at the previous batch of cooking lessons to decide how to give credit for this batch. This prevents the coach from getting confused by its own immediate reactions. It's like a teacher grading a test based on the rubric they wrote last week, not the one they just scribbled down while grading.
Segment-Level Credit: Instead of giving the whole team a score, GRPB breaks the cooking process into segments: "Chopping," "Simmering," "Seasoning." It figures out which specific segment contributed most to the win or loss. If Team C chopped onions perfectly, that specific step gets the credit. If Team B burned the garlic, that specific step gets the blame.
Safety Caps: To prevent the AI from getting too excited or too depressed, the system puts "caps" on how much credit or blame any single step can receive. It also uses a "reliability gate," meaning it only starts giving out credit once it's sure its judging system is working correctly.

What They Found

The researchers built a 9-billion-parameter AI model named Crayotter and trained it using this new method. They tested it in a simulated environment with realistic editing tasks, ranging from simple cuts to complex, multi-step revisions.

The results showed that GRPB worked better than other methods:

Better Editing: The videos produced by the GRPB-trained model were preferred by human judges more often than those from models trained with standard methods. In a blind test, the GRPB model won against the "Base" model (the original AI) 67.1% of the time.
Smarter Credit: When the researchers tested to see if the AI knew why it made a good video, GRPB was much better at pinpointing the specific editing steps that mattered. It correctly identified the "winning" segment 20.8% of the time, compared to only 11.9% for other methods.
Beating the Pros: On a standard benchmark called AgenticVBench, the 9B Crayotter model ranked third among all the systems tested, beating several expensive, proprietary systems from big tech companies. It scored a 15.7% average, with a particularly strong 23.0% score on the "Repurpose" task (which is very similar to video editing).

Why It Matters

This paper suggests that for complex, creative tasks where the "right answer" is subjective, we don't need a perfect score. We just need a way to compare different attempts and figure out which specific steps led to the better outcome. By using a system that compares similar attempts, delays its own judgment to avoid bias, and carefully distributes credit to specific parts of the process, AI can learn to be a much better editor.

The authors are careful to note that this is a specific solution for tasks like video editing, where there are clear stages and alternative outcomes. They don't claim this solves every AI problem, but they show that for these kinds of "long-horizon" creative jobs, breaking the problem down into local, comparable preferences is a powerful way to teach machines how to create. The code and data for their experiments are publicly available, inviting others to try this "lagged credit" approach on their own creative AI projects.

Technical Summary: Crayotter and Group-Relative Preference Backpropagation (GRPB)

1. Problem Statement

Long-horizon video editing agents face a fundamental challenge in learning from feedback: the quality of the final product is subjective, admits multiple valid solutions, and is only observable after a sequence of interdependent decisions (source analysis, clip retrieval, timeline construction, rendering, and revision).

Current approaches struggle with two primary issues:

Ambiguous Global Objectives: A global scalar reward is ill-suited for editing because quality dimensions (narrative continuity, pacing, stylistic fidelity) vary by task and judge. An absolute score entangles product quality, task difficulty, and judge-scale variation, making cross-task calibration meaningless.
Delayed and Diffuse Credit: Execution traces provide verifiable feedback on tool completion and artifact validity but fail to capture semantic selection or narrative quality. These qualities are only observable after rendering. Standard reinforcement learning (RL) methods struggle to assign credit from a final product judgment back to specific, early-stage editing decisions without introducing noise or bias.

Existing methods either rely on externally verifiable terminal conditions (which editing lacks) or broadcast a single trajectory-level advantage to all actions, failing to localize credit to the specific semantic segments that caused the outcome.

2. Methodology: Group-Relative Preference Backpropagation (GRPB)

The authors introduce GRPB, a framework that converts subjective, delayed outcomes into structured, task-local preference learning. The method operates within the Crayotter editing environment, which provides segmented traces, rendered outcomes, and process diagnostics.

Core Mechanisms

GRPB decomposes the learning problem into three distinct stages:

A. Task-Local Preference Construction
Instead of regressing a global quality score, GRPB samples a group of rollouts ( $G_q$ ) for the same request, materials, target duration, and constraints. A multimodal judge ranks these final videos ( $J_k$ ).

Zero-Sum Rank Advantage: The judge scores are converted into a pairwise win/loss advantage ( $A^{rank}_k$ ) within the group. This statistic is zero-sum ( $\sum A^{rank}_k = 0$ ) and lies in $[-1, 1]$ , making it invariant to score offsets, scale changes, and cross-task calibration.

B. Lagged Segment Allocation
The core innovation is mapping the trajectory-level preference to specific editing segments (contiguous events sharing an editing-stage label).

Sparse Bradley-Terry Allocator: A sparse linear model predicts the utility of a rollout based on the mean of its segment feature vectors ( $\hat{z}_k = w^\top \bar{f}_k$ ). The weights $w$ are trained to minimize the Bradley-Terry loss on the within-group rankings.
Lagged Update: Crucially, the allocator uses weights ( $w^-$ ) frozen before observing the current batch. Policy credit is computed using $w^-$ , and only then is $w$ updated. This prevents the model from memorizing current labels and immediately returning them as credit to the same batch.
Reliability Gating: Early allocator estimates are gated by a reliability coefficient based on warm-up observations and accuracy. Credit is disabled until the allocator demonstrates sufficient predictive accuracy.

C. Bounded Credit Redistribution
The zero-sum rank advantage is redistributed to segments using a "water-filling" algorithm with strict constraints:

Signed Capped Allocation: Positive budgets favor high-scoring segments; negative budgets penalize low-scoring ones. A per-segment absolute cap ( $C$ ) prevents any single segment from absorbing the entire budget.
Exact Conservation: The sum of allocated credits for a rollout exactly equals its rank advantage budget ( $\sum c_{k,m} = b_k$ ).
Semantic Placement: Credits are assigned to the final trainable position of a semantic segment, not distributed by token count, ensuring long verbal spans do not receive disproportionate preference mass.

D. Policy Optimization
The allocated segment credits are added to the process rewards at segment endpoints. The policy is optimized using PPO with GAE (Generalized Advantage Estimation), utilizing a learned value baseline. Unlike GRPO (Group Relative Policy Optimization), which shares a trajectory-level scalar across all actions, GRPB uses the group only to construct the budget, while the critic and GAE handle the step-varying advantages derived from localized segment rewards.

3. Key Contributions

Formulation: The paper formulates long-horizon video editing as a task-local artifact preference credit assignment problem, separating verifiable process feedback from subjective final-product assessment.
Algorithm (GRPB): A novel method combining zero-sum within-task rank advantages with a lagged, reliability-gated, and capped segment allocator. It ensures exact return conservation and prevents current judgments from directly shaping the same rollout group.
Dataset and Evaluation: The authors manually constructed a project-disjoint, horizon-stratified suite of 70 realistic editing tasks (12 normal, 12 medium, 46 long-horizon) with project-disjoint splits to prevent data leakage.

4. Experimental Results

The authors evaluated a 9B parameter model (Crayotter) trained with GRPB against several baselines and external benchmarks.

Matched Baselines (AgenticVBench):
- GRPB achieved the highest average score (15.7%) on the AgenticVBench, outperforming Process-PPO (13.7%) and Terminal Rank PPO (12.0%).
- Specifically, GRPB excelled in the "Repurpose" pillar (23.0%), a metric highly aligned with editing tasks.
- Ablation studies showed that removing the lag mechanism or reliability safeguards caused performance to drop below Process-PPO, confirming the necessity of guarded, differentiated credit.
Credit Localization:
- In controlled interventions where a single segment was degraded, GRPB achieved 20.8% Top-1 localization (identifying the degraded segment) and 85.7% pairwise accuracy in ordering, significantly outperforming uniform allocation (11.9%) and terminal broadcast (1.8%).
External Benchmarking:
- On the full AgenticVBench (100 disjoint tasks), Crayotter-9B ranked third among listed systems, exceeding several proprietary alternatives and matching Gemini 3.1 Pro on the Repurpose task.
Human Evaluation:
- In blinded human preference tests, GRPB achieved positive net win rates against all opponents, including a 34.2% net win rate over the base model and 8.0% over Process-PPO.

5. Significance and Claims

The paper claims that GRPB provides a practical approach to learning from subjective, delayed outcomes in multistage production tasks. By converting ambiguous global scores into controlled, within-task ordinal comparisons and localizing credit through a lagged allocator, the method enables agents to improve both editing behavior and rendered product quality without requiring cross-task score calibration.

The authors position this work as a foundation for other subjective, multi-solution, multistage production tasks, demonstrating that task-local comparison and conservative credit allocation can effectively bridge the gap between final-artifact judgment and intermediate policy optimization. The resulting Crayotter model demonstrates that open-source agents can compete with, and in some cases surpass, proprietary systems in complex video editing domains.