💻 computer science

Enhancing Software Maintenance: A Learning to Rank Approach for Co-changed Method Identification

This paper proposes a learning-to-rank approach that leverages source code features and pull request history to accurately identify and rank co-changed methods, demonstrating that a Random Forest model significantly outperforms existing baselines in managing software dependencies across large-scale Java projects.

Original authors: Yiping Jia, Safwat Hassan, Ying Zou

Published 2026-06-19

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Yiping Jia, Safwat Hassan, Ying Zou

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Problem: The "Domino Effect" in Code

Imagine you are managing a massive, complex city (a software system). Sometimes, you need to fix a pothole on Main Street. But because of how the city is built, fixing that one pothole might accidentally cause a traffic light on 5th Avenue to malfunction, or a sewer pipe in the basement to leak.

In software, these are called co-changed methods. They are different pieces of code that, even if they look unrelated and live in different "neighborhoods" (files or packages), tend to change together over time. If a developer fixes one but forgets the other, the whole system can crash or develop bugs.

The problem is that these connections aren't always obvious. You can't always see them by looking at the blueprints (the code structure). Sometimes, two pieces of code are linked only because of the "history" of how the team has worked in the past.

The Old Way vs. The New Way

The Old Way (The "Commit" Mistake):
Previous tools tried to spot these connections by looking at individual "commits" (small, daily updates to the code).

Analogy: Imagine trying to figure out who are best friends by only looking at who sat at the same lunch table for exactly 15 minutes on a single Tuesday. You might miss the fact that they go to the gym together every morning, or you might think two people are friends just because they happened to grab coffee at the same time once by accident.
The Flaw: This method was too noisy. It missed connections that happened over a longer period and included too many false alarms.

The New Way (The "Pull Request" View):
This paper proposes looking at Pull Requests (PRs) instead. A PR is like a "package" of changes that a team reviews and approves before merging it into the main system.

Analogy: Instead of looking at a 15-minute lunch, we look at the entire week's meal plan. If two people consistently order the same complex meal together every week, they are likely a team. This gives a much clearer picture of who actually works together.

The Solution: CoRanker (The "Smart Matchmaker")

The authors built a tool called CoRanker. Think of it as a smart matchmaker for code.

It Learns from History: Instead of using rigid rules (like "if they are in the same file, they are related"), CoRanker uses Machine Learning (specifically a "Learning-to-Rank" approach). It studies thousands of past Pull Requests to learn patterns.
It Weighs Many Clues: When you change one piece of code, CoRanker asks: "Who else is likely to need changing?" It looks at:
- History: Have these two changed together before? (The strongest clue).
- Location: Are they in the same folder?
- People: Did the same developers write or edit them?
- Meaning: Do they do similar things? (Even if the code looks different).
It Ranks the Answers: It doesn't just give a giant, confusing list of 1,000 possibilities. It acts like a search engine, putting the most likely candidates at the very top of the list so the developer only has to check the top 5.

What They Found (The Results)

The researchers tested this on 150 different software projects (a huge amount of data, like reading millions of pages of code).

The Best Model: They tried many different "matchmaking" algorithms. The winner was a Random Forest model. Think of this as a committee of 300 different experts voting on who the best match is. This method was significantly better than all the others.
Beating the Competition: CoRanker was much better than existing tools. It outperformed the next best method by a wide margin (up to 573% better in some tests).
The "LLM" Surprise: The researchers also tried using a fancy Large Language Model (like a super-smart AI chatbot trained on code) to guess the connections.
- The Result: The AI was actually worse than the simpler, history-based tool.
- Why? The AI is great at writing new code, but it struggled to understand the specific "history" of how these two pieces of code have evolved together over years. It's like asking a genius who just moved to town to guess who the local best friends are; they don't know the history yet.
How Often to Retrain: The tool works best if you update its memory every two months. If you wait too long (more than 60 days), the "old news" in its memory starts to confuse it, and it makes worse predictions.

Why This Matters

This tool helps developers avoid the "Domino Effect."

For the Developer: When you fix one bug, the tool whispers, "Hey, don't forget to check this other file too, or you'll break something."
For the Team: It helps them understand the hidden structure of their software, revealing that two distant parts of the code are actually best friends.

Summary

The paper introduces CoRanker, a smart system that predicts which parts of a software project need to be changed together. By looking at the history of "Pull Requests" (big batches of changes) instead of tiny daily updates, and by using a learning algorithm that weighs history, location, and authorship, it helps developers find hidden connections. It works better than old methods and even better than fancy AI chatbots for this specific task, provided it is updated every couple of months.

Technical Summary: Enhancing Software Maintenance via a Learning-to-Rank Approach for Co-changed Method Identification

Problem Statement
As software systems grow in complexity, accurately identifying all necessary modifications to implement a specific change becomes increasingly difficult. A critical challenge is detecting "co-changed methods"—methods that frequently evolve together over time, often indicating a strong evolutionary coupling even in the absence of explicit code dependencies. Existing approaches suffer from significant limitations:

Granularity Issues: Commit-level analysis often misses co-changes that span multiple commits within a single Pull Request (PR) or includes coincidental changes.
Scope Limitations: Clone-based detection overlooks relationships between non-cloned code.
Prioritization Deficits: Association rule mining often generates large, unordered result sets with high false-positive rates, lacking mechanisms to prioritize the most relevant candidates.
Heuristic Rigidity: Existing ranking methods rely on fixed scoring functions that fail to adaptively combine heterogeneous features (historical, structural, semantic).

To address these gaps, the authors propose CoRanker, a framework that shifts the detection granularity from the commit level to the Pull Request (PR) level and employs a Learning-to-Rank (LtR) approach to predict and rank likely co-changed methods.

Methodology
CoRanker is a supervised learning framework designed to rank candidate methods based on their likelihood of being co-changed with a query method. The methodology involves four main stages:

Data Collection & Project Selection:
- The study utilizes 150 open-source Java projects from GitHub, totaling 41.5 million lines of code (LOC) and 634,216 PRs.
- Projects were selected based on activity (active for >2 years, updated in last 6 months), volume (>200 merged PRs), and structural integrity (representing actual software systems).
Feature Extraction:
The model utilizes ten features derived from source code and version control history to represent the relationship between a query method ( $M_q$ ) and a candidate method ( $M_c$ ):
- Historical Features: Number of co-changes (frequency of joint modification in PRs) and Authors similarity (Jaccard similarity of contributor sets).
- Static/Structural Features: Semantic similarity (via CodeBERT embeddings), File path similarity, Code dependency (call relationships), Hierarchy similarity (shared superclass), Code clone score (via NiCad), Package similarity, Argument type similarity, and Argument name similarity.
- Redundancy Handling: "Package similarity" was removed due to high correlation with "Path similarity" to ensure generalizability.
Data Preparation & Labeling:
- The dataset is constructed using a time-separation approach to prevent data leakage. Features are extracted from a historical window (e.g., 12 months), and labels are generated based on co-change occurrences in a subsequent window (e.g., 6 months).
- Labels represent the frequency of co-changes between method pairs in the labeling period.
- Ranking lists are constructed for each query method, excluding lists where all candidates have zero co-change history.
Model Training & Evaluation:
- Algorithms: The study evaluates nine LtR models across point-wise (Linear Regression, MART, Random Forest, DNN), pair-wise (RankNet, LambdaMART), and list-wise (Coordinate Ascent, AdaRank, ListNet) categories.
- Baselines: CoRanker is compared against six baselines: Support ranking (frequency-based), File proximity, Code clone detection (NiCad), FCP2Vec, and two StarCoder 2 configurations (Scalar and Binary, in both zero-shot and history-aware settings).
- Metrics: Performance is measured using Normalized Discounted Cumulative Gain (NDCG@k) and Precision@k for $k \in \{1, 3, 5, 10\}$ .

Key Results

Model Performance: The Random Forest (RF) model emerged as the superior LtR algorithm, outperforming other LtR models by 2.5%–12.8% in NDCG@5. It achieved a mean NDCG@5 of 0.8394 and Precision@5 of 0.7418.
Comparison with Baselines: CoRanker (using RF) significantly outperformed all six baselines, with improvements in NDCG@5 ranging from 4.7% to 573.5%. Notably, it outperformed the best baseline (Support ranking) by 4.7% in NDCG@5 and 10.2% in NDCG@1.
LLM Performance: While StarCoder 2 (15B parameters) was evaluated, it consistently underperformed compared to the feature-based RF model. Even with historical context provided in prompts, the LLMs failed to match the heuristic support-ranking baseline, suggesting that current pre-trained code models struggle to capture evolutionary coupling patterns without explicit feature engineering.
Feature Importance: Permutation importance analysis revealed that the Number of co-changes is the dominant feature (importance ~0.38). Path similarity and Authors similarity also contributed positively. Conversely, Argument name similarity had a slight negative impact, and semantic similarity (CodeBERT) showed minimal contribution.
Temporal Dynamics: Models trained on longer historical data (90–180 days) demonstrated more consistent performance than those trained on shorter periods (30 days). However, prediction accuracy began to decline after 60 days of testing data, suggesting a need for bi-monthly model retraining to maintain optimal performance.

Significance and Contributions
The paper claims the following contributions:

Framework Proposal: Introduction of CoRanker, a learning-to-rank framework that integrates heterogeneous features to prioritize co-changed methods at the PR level, overcoming the limitations of commit-level and heuristic approaches.
Large-Scale Empirical Evaluation: A comprehensive study on 150 real-world Java projects, providing a robust dataset and replication package.
Performance Validation: Demonstration that LtR models, specifically Random Forest, significantly outperform existing baselines and LLM-based approaches in identifying co-changed methods.
Feature Insights: Empirical evidence that while historical co-change frequency is the primary predictor, the integration of structural and semantic features via learning models substantially improves early-rank recommendation quality.
Practical Deployment Guidelines: Insights into the operational requirements of such a tool, including the necessity of periodic retraining (every 2–3 months) and the feasibility of integrating the system into IDEs or CI pipelines with low runtime overhead (inference <1 second per query).

The authors conclude that CoRanker offers a practical tool for software teams to manage complex dependencies, reduce the risk of missing necessary modifications, and improve overall software maintenance quality. They acknowledge limitations regarding the scope (Java/GitHub only) and the lack of user studies, proposing future work to generalize to other languages and integrate directly into development environments.