💻 computer science

EasyVideoR1: Easier RL for Video Understanding

This paper introduces EasyVideoR1, a comprehensive and efficient reinforcement learning framework specifically designed to overcome the computational and evaluation challenges of training large vision-language models on video understanding tasks through optimized preprocessing, a modular reward system, and a mixed training paradigm.

Original authors: Chuanyu Qin, Chenxu Yang, Qingyi Si, Naibin Gu, Dingyu Yao, Zheng Lin, Peng Fu, Nan Duan, Jiaqi Wang

Published 2026-04-21

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Chuanyu Qin, Chenxu Yang, Qingyi Si, Naibin Gu, Dingyu Yao, Zheng Lin, Peng Fu, Nan Duan, Jiaqi Wang

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a brilliant student who is amazing at reading books and solving math problems. Now, you want to teach this student to understand movies.

Movies are tricky. They are long, they move, they have sound, and they change every second. If you try to teach the student by showing them a raw video file every single time they ask a question, it's like trying to cook a meal by chopping fresh vegetables from the garden for every single bite. It's slow, exhausting, and you spend more time chopping than eating.

This is the problem the paper "EasyVideoR1" solves. It's a new toolkit designed to make teaching AI models how to understand videos much faster, smarter, and more effective.

Here is the breakdown of how it works, using some everyday analogies:

1. The "Pre-Chopped Ingredients" (Offline Caching)

The Problem: In old systems, every time the AI looked at a video, the computer had to stop, decode the video file, cut it into frames, and resize the pictures. It did this three times during a single training session. It was like a chef re-chopping the same onions for every single step of the recipe.

The EasyVideoR1 Solution: They created a "Pre-Chopped Ingredients" system. Before training even starts, they process all the videos once and save them as ready-to-use "tensors" (digital ingredients).

The Result: When the AI needs to learn, it just grabs the pre-chopped ingredients from the fridge. No more chopping! This made the training 1.5 times faster, saving a massive amount of time and computer power.

2. The "Mixed Classroom" (Joint Image & Video Training)

The Problem: Usually, AI models learn from pictures (static) and videos (moving) separately. But a model that only sees movies might forget how to look at a single photo, and vice versa. Also, videos are huge, so you can't fit as many of them on a computer's memory as you can photos.

The EasyVideoR1 Solution: They built a "Mixed Classroom" where the AI learns from both photos and movies at the same time.

The Analogy: Imagine a gym where you can lift heavy weights (videos) and do quick cardio (images) in the same workout. The system is smart enough to know: "Okay, for this video, let's lower the resolution slightly to fit more frames, but for this photo, let's keep it super high-definition." They don't interfere with each other; they actually help the AI get stronger in both areas.

3. The "Study Buddy & The Exam" (Offline + Online Training)

The Problem: Teaching AI with Reinforcement Learning is like a student taking a test. If the student has never seen the material before (cold start), they fail the first few tests, get discouraged, and learn slowly.

The EasyVideoR1 Solution: They introduced a "Hybrid Study Method."

The Analogy: Instead of forcing the student to guess the answers from scratch every time, they give them a "Cheat Sheet" (Offline Data) of high-quality examples they already know are correct.
The AI studies these perfect examples and tries to solve new problems on its own (Online Exploration) at the same time. This helps the AI learn difficult tasks much faster because it has a solid foundation to build on.

4. The "Super-Fast Grader" (Asynchronous Evaluation)

The Problem: Checking how well an AI understands 22 different video benchmarks is like grading 1,000 essays one by one while the teacher is also trying to write a new essay. The computer sits idle waiting for the video to load, then waits for the AI to think, then waits for the grade. It's a traffic jam.

The EasyVideoR1 Solution: They built a "Conveyor Belt Grading System."

The Analogy: While the computer is loading the next video (Loading Stage), it is simultaneously grading the current video (Decoding Stage) and preparing the next one (Prefill Stage).
Nothing waits for anything else. The computer is always working. This made their evaluation process 6 to 7 times faster than standard methods.

The Big Win: What Did They Achieve?

They took a smart AI model (Qwen3-VL-8B) and gave it a "boot camp" using this new toolkit.

Before: The model was good, but not great at reasoning through complex video puzzles.
After: After just 20 hours of training on 32 powerful GPUs, the model became smarter than its "Thinking" version (a version designed to think longer) on many video tests.
The Takeaway: You don't necessarily need a bigger, slower model to get better results. You just need a better way to train it.

In a Nutshell

EasyVideoR1 is like upgrading a cooking school. Instead of making students chop vegetables from scratch for every dish, they give them pre-prepped ingredients, let them practice with both simple and complex recipes at the same time, and use a conveyor belt to grade their work instantly. The result? The students (AI models) learn to cook (understand video) faster and better than ever before.

1. Problem Statement

While Reinforcement Learning from Verifiable Rewards (RLVR) has significantly improved the reasoning capabilities of Large Language Models (LLMs), extending this success to Video Understanding remains largely unexplored and technically challenging. Existing open-source RL frameworks (e.g., EasyR1, OneThinker) are primarily optimized for text and image modalities and face critical bottlenecks when applied to video:

Computational Overhead: Video data requires expensive preprocessing (decoding, frame sampling, resizing). Existing frameworks redundantly decode and preprocess video data up to three times across different pipeline stages (dataset loading, rollout generation, and actor training), creating a severe CPU-bound I/O bottleneck.
Task Diversity & Reward Complexity: Video understanding spans a wide spectrum of tasks (e.g., temporal grounding, object tracking, OCR, spatial segmentation) requiring specialized reward logic that current frameworks lack.
Evaluation Difficulties: Reproducible evaluation on video benchmarks is sensitive to numerous hyperparameters (frame sampling strategy, resolution, FPS). Existing tools often fail to faithfully reproduce official scores or suffer from slow inference due to synchronous CPU-GPU workflows.
Data Scarcity: High-quality video RL datasets are scarce compared to image/text, making pure on-policy training inefficient and prone to cold-start issues.

2. Methodology: EasyVideoR1 Framework

EasyVideoR1 is an open-source, end-to-end RL training framework built upon EasyR1 and veRL, specifically engineered to address the unique challenges of video modalities. Its architecture focuses on three core dimensions:

A. Video-Friendly Optimization (Efficiency)

Offline Preprocessing & Tensor Caching: The framework decouples video preprocessing from the training loop. Videos are decoded, resampled, and resized offline into .pt cache files.
- Mechanism: During training, workers load cached tensors locally rather than decoding raw video files. Metadata (frame rate, sampling indices) is propagated alongside the cache to ensure consistency across stages.
- Impact: Eliminates redundant decoding, reducing I/O bottlenecks and preventing the "three-pass" decoding issue found in prior work.
Mixed-Modality Pipeline Adaptation:
- Dummy Tensors: To handle mixed image-video batches under Fully Sharded Data Parallel (FSDP), the system generates zero-valued dummy tensors for missing modalities. This ensures all encoder branches participate in the forward pass, preventing gradient synchronization failures.
- Independent Budgets: It supports independent configuration of pixel budgets for images (image_max_pixels) and videos (video_max_pixels, video_max_frames), allowing optimal resource allocation for each modality.

B. Research-Friendly Interfaces (Algorithm & Data)

Hybrid Offline-Online Training: To mitigate cold-start issues and leverage scarce high-quality data, EasyVideoR1 implements a mix-policy interface. It allows a single training iteration to combine pre-collected offline trajectories (from stronger models or curated datasets) with $n-1$ online rollouts generated by the current policy. This is controlled via a simple flag without modifying the core GRPO algorithm.
Joint Image-Video Training: The framework supports batching static images and video clips simultaneously. It unifies the data schema and enforces strict consistency checks (raising exceptions if placeholder tokens do not match visual features) to prevent data corruption during long training runs.
Task-Aware Reward System: A modular reward library with a unified dispatcher handles 11 distinct task types (e.g., Multiple Choice, Temporal Grounding, OCR, Math). Each task type has a dedicated scoring module (e.g., 1D IoU for temporal grounding, ROUGE for open-ended QA).

C. Asynchronous Multi-Benchmark Evaluation

AsyncLLMEngine Integration: The evaluation pipeline utilizes vLLM's asynchronous engine to eliminate CPU-GPU serialization.
- Three-Stage Overlap: It runs IO (loading cached frames), Prefill, and Decode stages concurrently. While the IO stage loads the $(N+2)$ -th sample, the Prefill stage processes the $(N+1)$ -th, and the Decode stage generates tokens for the $N$ -th.
- Chunked Prefill: Prevents long video sequences from monopolizing the GPU by partitioning inputs into fixed-size token chunks.
Benchmark Coverage: Supports 22 mainstream video benchmarks across six categories (General, Long Video, Reasoning, STEM, Spatial, Streaming) with protocols adapted to reproduce official accuracy scores.

3. Key Contributions

Complete Video RL Pipeline: A full-stack solution featuring offline preprocessing and tensor caching, yielding a 1.47× throughput improvement by eliminating redundant video decoding.
Comprehensive Reward System: A modular library covering 11 video/image problem types with unified routing, enabling flexible extension to new tasks.
Mixed Training Paradigm: A novel hybrid regime combining curated offline trajectories with on-policy exploration, specifically beneficial for challenging tasks with sparse reward signals.
Joint Image-Video Training: Engineering support for mixed-modality micro-batches with independent resolution budgets, allowing image data to reinforce video temporal reasoning.
High-Throughput Evaluation: An asynchronous evaluation framework covering 22 benchmarks, achieving 6–7× speedup over vanilla inference while ensuring reproducible accuracy.

4. Experimental Results

The authors evaluated EasyVideoR1 using Qwen3-VL-8B-Instruct trained on 32 H200 GPUs for approximately 20 hours (200 GRPO steps) on ~100K video samples.

Performance Gains: The RL-trained model surpassed its "Thinking" variant (Qwen3-VL-8B-Thinking) on multiple benchmarks without requiring additional inference-time reasoning overhead.
- Average Improvement: +2.3% accuracy across 10 diverse benchmarks.
- Reasoning Tasks: Significant gains on Video-Holmes (+6.6%) and VideoMathQA (+6.7%), demonstrating enhanced deliberative reasoning.
- General Understanding: Consistent improvements on Video-MME (+2.1), MVBench (+3.5), and LVBench (+0.7).
Efficiency Gains:
- Throughput: Cache-based loading achieved 1.47× speedup in tokens per second (797 → 1,175 tok/s).
- Latency: Reduced average step time from 194.5s to 131.9s.
- Breakdown: Rollout generation time decreased by 1.52×, and reference model forward pass time decreased by 2.85× due to the elimination of redundant decoding.

5. Significance

EasyVideoR1 addresses a critical gap in the multimodal AI ecosystem by providing the first systematic, efficient, and reproducible framework for RL post-training on video understanding.

Lowering Barriers: By solving the engineering complexities of video I/O, caching, and mixed-modality training, it lowers the entry barrier for researchers to experiment with RL in video domains.
State-of-the-Art Performance: It demonstrates that RL can effectively unlock the reasoning potential of video-language models, even surpassing models explicitly designed with "thinking" mechanisms.
Community Resource: The framework is fully open-sourced, supporting a wide range of backbones (Qwen2/2.5/3/3.5-VL) and algorithms (GRPO, DAPO, etc.), fostering further innovation in video reasoning and temporal understanding.