💬 NLP

Change2Task: From Repository Changes to Executable Coding Agent Tasks and Environments

The paper introduces Change2Task, a system that automatically converts merged pull requests into verified, executable coding agent tasks by reconstructing historical repository states and validating them on modern codebases, thereby significantly increasing the supply of training data while reducing setup costs and improving task recovery rates compared to existing baselines.

Original authors: Haomin Qi, Xingliang Wang, Xuanqi Gao, Baihui Sang, Xin Zhang, Minghua Ma, Pengfei Gao, Yu Kang, Qingwei Lin, Saravan Rajmohan, Dongmei Zhang, Qi Zhang

Published 2026-07-31

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Haomin Qi, Xingliang Wang, Xuanqi Gao, Baihui Sang, Xin Zhang, Minghua Ma, Pengfei Gao, Yu Kang, Qingwei Lin, Saravan Rajmohan, Dongmei Zhang, Qi Zhang

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine a world where computers are learning to be software engineers. These "coding agents" are like digital apprentices that don't just write a single line of code and stop; they can open a project, read the files, fix a bug, run tests, and try again if they fail. But here's the catch: to get really good at this, these apprentices need to practice on real-world problems. They need a massive library of "exercises" where they can try to fix a broken feature or add a new one, and then get a clear "pass" or "fail" grade.

The problem is that creating these practice exercises is incredibly hard and expensive. Usually, to make a test, you have to build a perfect, isolated computer environment (like a digital sandbox) for every single problem. If you want 1,000 different coding challenges, you might need to build 1,000 different sandboxes, which takes up huge amounts of storage space and time. It's like trying to teach someone to cook by building a brand-new, fully stocked kitchen for every single recipe you want them to learn. It's so much work that we run out of practice problems, and the agents can't get the training they need to become experts.

This is where a new idea called Change2Task comes in. Think of it as a magical time machine for code. Instead of building a new kitchen for every recipe, Change2Task looks at the history of a real software project—like a giant logbook of every time a human developer fixed a bug or added a feature in the past. It takes those old, real-life stories and asks: "Can we recreate this exact same challenge, but on the current version of the software?"

The system works like a clever editor. It finds a "merged" pull request (which is just a fancy term for a change that was already accepted and added to the code) and tries to reverse-engineer it. It asks the computer to "undo" that change to create a broken state, then sets a goal for the coding agent to fix it again. But here's the magic trick: instead of starting from scratch, it does this on a modern, healthy version of the software that is already running. If the software has changed a lot since the original fix, Change2Task uses three different strategies to map the old fix onto the new code:

Patch Reversal: If the code hasn't changed much, it just flips the old fix backward.
Code Mapping: If the code moved around, it finds the new location of the old code and swaps it out.
Agent Reconstruction: If the code is totally different, it uses a smart AI agent to figure out how to recreate the problem in the new environment.

The researchers tested this on five common types of coding tasks: fixing bugs, adding new features, writing new tests, updating old tools (APIs), and patching security holes. They started with 1,130 real-world changes from public software projects. The system successfully turned 900 of them into working, verified practice tasks. That's a success rate of about 80%.

What makes this really exciting is how much it saves. By reusing the same modern "kitchen" (the software environment) for multiple different practice problems, they reduced the time needed to set up environments by 58.4% and cut the storage space needed by 71.2%. In fact, they managed to create 29.2% more verified tasks than a previous method that just tried to copy old fixes directly.

The team also checked if these recreated tasks were fair. They asked different AI coding agents to solve the problems on both the original historical version and the new modern version. The results were almost identical: the agents got the same "pass" or "fail" grade 98% of the time. This proves that the new tasks aren't just random guesses; they are faithful recreations of real developer work.

In short, Change2Task doesn't just find more practice problems; it finds a way to make the ones we already have reusable and efficient. It turns the history of software development into a renewable resource for training the next generation of AI coders, proving that we don't need to build a new world for every lesson—we just need to know how to look at the old one with fresh eyes.

Technical Summary: Change2Task

Problem Statement

Scaling coding agents requires a continuous, high-quality supply of executable data for training, benchmarking, and evaluation. Each task unit must couple a realistic software state with a specification, development tools, and reliable verification. Current approaches face significant bottlenecks:

Cost and Scale: Producing executable environments is expensive. Systems like daVinci-Env and SWE-Universe require massive infrastructure and storage to maintain distinct snapshots for individual tasks, leading to high construction costs and storage pressure.
Data Limitations: Historical benchmarks (e.g., SWE-bench) bind tasks to original, often outdated repository revisions, limiting their applicability to modern codebases. Conversely, synthetic task generation (e.g., SWE-smith) can produce instances that lack genuine developer intent or maintenance context.
The Gap: There is a need to derive multiple verified, developer-grounded tasks from a single maintained repository environment to reduce redundant setup efforts while expanding the diversity of executable data.

Methodology

Change2Task is a framework designed to convert merged Pull Requests (PRs) from repository history into verified coding agent tasks on healthy, modern descendant revisions of the same repository. The system operates on the lifecycle $V_{pre} \xrightarrow{P_s} V_{post}$ (historical state change) and reconstructs it as $H \xrightarrow{D} C \xrightarrow{G} H'$ (modern base to task state to restored state).

The process involves four key stages:

1. Deriving Task Evidence

The system extracts evidence from a merged PR, including its description, implementation patch, and executable checks. This defines:

Target Checks: Conditions to be reconstructed.
Regression Checks: Surrounding behaviors that must remain intact.
Source Change Profile: A record of affected components and edit extent.
Candidates are excluded if conditions are ambiguous, checks are unexecutable, or changes cannot be isolated.

2. Establishing the Modern Task Base

For each eligible PR, a fixed, healthy descendant revision ( $H$ ) is selected from the same repository lineage. This base must support a clean checkout, runtime dependencies, and executable checks. The system ensures that mapped target and regression checks pass on $H$ before proceeding.

3. Constructing the Task State

Change2Task employs a three-level adaptive construction strategy to generate the task patch ( $D$ ) that transforms the healthy base ( $H$ ) into the task state ( $C$ ):

Level 1: Patch Reversal: Directly reverses the source patch if it remains compatible with the modern context.
Level 2: Code Mapping: If direct replay fails, the system locates the historical post-change block in the modern file and replaces it with its pre-change form, allowing for indentation normalization and conservative local transformations.
Level 3: Agent Reconstruction: If structural correspondence is insufficient, a construction agent (using PR evidence and modern context) proposes scoped candidate patches. These are filtered by syntax, scope, and lifecycle checks, then ranked by source change profile fidelity.

4. Lifecycle and Scope Validation

Every candidate undergoes validation across three states:

Healthy ( $H$ ): Target and regression checks pass.
Task ( $C$ ): Target checks expose the specific task condition (e.g., a bug or missing feature), while regression checks continue to pass.
Restored ( $H'$ ): After applying a restoration patch ( $G$ ), both check sets pass, confirming the system can return to a healthy state.

The system also enforces Scope and Fidelity gates to ensure the modern task does not trivialize the original change (scope collapse) or require unrelated broad edits (scope inflation).

Key Contributions

Change2Task Framework: A general tool that transfers developer-grounded maintenance changes to current code, verifying provenance, behavior, scope, and restoration.
Adaptive Construction: A three-tiered approach (Reversal, Mapping, Reconstruction) that escalates complexity only when necessary, maximizing the recovery of valid tasks from historical data.
Task Adapters: A unified interface supporting five distinct task families: Bug Fix, Feature Addition, Test Generation, API Migration, and Security Repair.
Efficiency Mechanism: By deriving multiple lightweight task variants from a single maintained base, the system significantly reduces repeated environment setup, storage, and manual construction efforts.

Experimental Results

The system was evaluated on 1,130 construction-eligible source changes across five task families.

Recovery Rate: Change2Task finalized 900 paired tasks, achieving a 79.6% overall verified task construction success rate.
- Bug Fix: 500 tasks recovered.
- Comparison: On a matched set of 621 Bug Fix candidates, Change2Task recovered 500 tasks, compared to 387 for the SWE-smith PR Mirror baseline (a 29.2% relative gain).
Fidelity: The corpus achieved a 0.894 task-weighted source change profile fidelity. Historical and reconstructed cases showed up to 98.0% matched outcome agreement under agent evaluation.
Agent Consistency: Across 3,600 matched agent-task pairs, the aggregate agreement between the Original (historical) and Change2Task (modern) branches was 89.7% (Cohen's $\kappa$ = 0.787). The paired solve-rate difference was negligible (0.0 points), indicating the modern tasks preserve the comparative evaluation signal of their historical anchors.
Resource Efficiency: Reusing 388 modern bases for 900 tasks (vs. preparing one base per task) resulted in:
- 58.4% reduction in total setup time.
- 71.2% reduction in storage requirements.
- 10.8% reduction in end-to-end expenditure.

Significance and Claims

The paper claims that Change2Task effectively transforms repository history from a passive record into a recoverable source of verified coding tasks. Its primary significance lies in:

Scalability: It provides a mechanism to expand the supply of executable data for coding agent training without the prohibitive costs of creating unique environments for every task.
Grounding: It preserves the "developer-grounded maintenance contract" of historical changes, ensuring tasks reflect real-world maintenance intents rather than synthetic artifacts.
Efficiency: By reusing modern bases, it drastically reduces the computational and storage overhead associated with large-scale agent evaluation.

The authors position Change2Task as complementary to existing environment preparation tools (like RepoLaunch), specifically addressing the phase after a base is executable, where the goal is to maximize the number of distinct, verified tasks derived from that investment.