💻 computer science

Project-wise Comparison of Software Birthmarks Using Weighted Partial Similarity

This paper proposes a project-wise software birthmark comparison framework that employs weighted aggregation and partial similarity mechanisms to robustly detect partial code reuse and mitigate false positives caused by small modules, demonstrating superior performance over existing methods across diverse open-source Java projects.

Original authors: Nikolay Fedorov, Akito Monden, Hiroki Inayoshi, Haruaki Tamada, Masateru Tsunoda

Published 2026-06-30

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Nikolay Fedorov, Akito Monden, Hiroki Inayoshi, Haruaki Tamada, Masateru Tsunoda

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a detective trying to solve a case of software plagiarism. Someone has taken a piece of code from an open-source project, tweaked it, and claimed it as their own. Your job is to prove they stole it.

In the past, detectives (researchers) looked at this problem one file at a time. They would compare File A from Project X with File B from Project Y. If they looked similar, they flagged it.

But real-world software is like a massive library, not a single book. A project might have thousands of files. Often, a thief only steals one or two chapters (modules) from a book and puts them into their own massive encyclopedia. If you compare the whole encyclopedia to the original, the stolen chapters get lost in the noise. Also, sometimes two completely different libraries happen to have a few generic words in common (like "the" or "and"), which can trick the detective into thinking they are the same book.

This paper introduces a new, smarter way to compare entire software projects to catch these thieves. Here is how they did it, explained simply:

1. The Problem: The "Needle in a Haystack" and the "False Alarm"

The authors identified two main headaches with the old methods:

The Needle in a Haystack (Partial Reuse): If a project has 1,000 files and only 10 were stolen, looking at the average similarity of all 1,000 files dilutes the evidence. The "stolen" signal gets drowned out by the "clean" files.
The False Alarm (Incidental Similarity): Small, generic files (like a simple "Hello World" or a basic utility function) might look similar just by chance. If you treat a tiny 5-line file the same as a massive 5,000-line file, the tiny one can create a false alarm, making two innocent projects look like twins.

2. The Solution: A Two-Step Detective Strategy

The authors proposed a new framework that acts like a smart filter. They didn't just look at the files; they looked at the weight of the files and ignored the noise.

Step A: The "Weight Scale" (Weighting)

Imagine you are comparing two baskets of fruit. One basket has a giant watermelon, and the other has a tiny grape.

Old Method: Counts the grape and the watermelon as "1 piece of fruit" each.
New Method: Realizes the watermelon is much more significant. It gives the watermelon a heavy "weight" and the grape a light "weight."

In their software, they assigned higher importance to larger code modules. If a small file looks similar to another small file, the system says, "That's probably just a coincidence; ignore it." But if a huge, complex file looks similar, the system pays close attention. This stops the "false alarms" caused by tiny, generic files.

Step B: The "Top 1%" Rule (Partial Similarity)

Imagine you are looking for a specific song in a playlist of 1,000 songs. You don't want to listen to the whole playlist to find the match; you just want to hear the top few songs that sound most like your target.

Old Method: Averages the similarity of every file pair between two projects.
New Method: Says, "Let's only look at the top 1% to 5% of the most similar file pairs."

By focusing only on the "best matches" and ignoring the rest, the system ignores the thousands of unrelated files that don't matter. This makes it much easier to spot the "needle" (the stolen code) even if it's a small part of a huge project.

3. The Experiment: Testing the New Detective

To prove this works, the researchers built a test lab:

The Test Subjects: They gathered 35 real-world Java projects (like media players, text editors, and testing tools) from GitHub.
The Setup: They treated different versions of the same project as "stolen" cases (since new versions are just old versions with changes). They treated different projects in the same category (e.g., two different media players) as "innocent" cases.
The Metric: They measured two things:
1. Resilience: Can it still find the "stolen" code even if the thief changed it?
2. Credibility: Can it correctly say "No, these two are different" when they are actually different?

4. The Results: The New Method Wins

The results were clear:

The new method (Weighting + Top 1% Focus) was significantly better than all existing methods.
It was very stable (consistent results) and rarely made mistakes.
Interestingly, they found that symmetry mattered. If you compare Project A to Project B, the score should be the same as comparing B to A. Their new method ensured this balance, which older methods failed to do.
They also found that Edit Distance (a way of measuring how many changes you need to make to turn one string into another) was the best tool for comparing the actual code snippets.

The Bottom Line

This paper doesn't just say "we found a better way to count code." It says: "To catch a thief who only stole a few pages from a library, you need to ignore the tiny, generic pages and focus only on the heavy, complex chapters that match."

By giving more weight to big files and only looking at the very best matches, this new framework makes it much harder for plagiarists to hide their theft in a sea of code, and much harder for innocent projects to be falsely accused.

Technical Summary: Project-wise Comparison of Software Birthmarks Using Weighted Partial Similarity

Problem Statement
Software birthmarks are intrinsic program characteristics used to detect plagiarism and unauthorized code reuse. While existing similarity measures are effective at the module level (e.g., comparing individual source or class files), they are insufficient for large-scale, real-world software ecosystems where systems consist of numerous modules. Two primary challenges arise when aggregating module-level comparisons to the project level:

Partial Reuse: In many cases, only a subset of modules within a project is reused, while the majority remains unrelated. Uniformly aggregating all module pairs dilutes the signal of actual reuse.
Incidental Similarity: Small or generic modules often exhibit high similarity by chance across unrelated projects. When treated equally with larger modules, these "noisy" matches can lead to false positives, undermining the credibility of the detection.

Current project-wise methods often fail to address these issues systematically, lacking robust mechanisms to distinguish between meaningful partial reuse and incidental noise.

Methodology
The authors propose a framework for project-wise birthmark comparison that aggregates module-level similarities using two complementary mechanisms: Weighted Aggregation and Partial Similarity.

Symmetric Aggregation (SA): The framework establishes a baseline symmetric aggregation method. Unlike previous asymmetric approaches, SA calculates the average of the highest similarity scores found for each module in Project A against Project B, and vice versa. This ensures the similarity score is independent of the input order.
Weight Assignment: To mitigate incidental similarity from small modules, the authors assign weights to modules based on their size (specifically, the number of $k$ -grams). A logarithmic transformation ( $W = \ln(K)$ ) is applied to compress the weight range, preventing extremely large modules from dominating the score while still reducing the influence of small, noisy modules. The final weighted similarity between two modules is the product of their raw similarity and the minimum of their normalized weights.
Partial Similarity: To address partial reuse, the framework focuses only on the top fraction ( $\alpha\%$ ) of the highest weighted similarity scores between module pairs. By ignoring the bulk of unrelated module pairs, this method emphasizes the most relevant reused components.
Evaluation Metrics: The performance is evaluated using Resilience (the ability to detect reused code despite modifications) and Credibility (the ability to distinguish independently developed projects). These are combined via their harmonic mean ( $H_{mean}$ ) to ensure a balance between detecting true positives and avoiding false positives.

Experimental Setup
The study utilizes a dataset of 35 open-source Java projects across 10 categories. Reuse cases are simulated by comparing different versions of the same project, while non-reuse cases are formed by comparing different projects within the same category. The experiments employ static $k$ -gram birthmarks and evaluate various module-wise similarity functions (e.g., Cosine, Jaccard, Edit Distance) and parameter configurations.

Key Results

Superiority of the Proposed Method: The proposed method (combining weighting and partial similarity with a scope of 1%) consistently outperforms existing approaches, including the baseline method by Lee et al. and previous aggregated similarity variants. It achieves the highest median $H_{mean}$ (0.9728) with low variance.
Necessity of Weighting: Weighting is critical. When partial similarity is applied without weighting, performance degrades significantly (even below random similarity in some configurations) because small modules dominate the top-ranked pairs. Weighting suppresses this noise, stabilizing the results.
Effectiveness of Partial Similarity: Focusing on a small subset of highly similar module pairs (e.g., top 1% or 5%) yields better results than considering all pairs. This confirms that ignoring unrelated modules is essential for detecting partial reuse.
Optimal Configurations: The best performance is achieved with small comparison scopes (1–5%), larger $k$ -gram sizes (3–4), and the Edit Distance similarity function, which demonstrated the highest accuracy and stability among the tested functions.
Symmetric Aggregation Baseline: Even without weighting or partial similarity, the symmetric aggregation formulation itself outperforms existing asymmetric methods, providing a strong foundation for project-wise comparison.

Significance and Claims
The paper claims to establish a robust framework for project-wise birthmark comparison that addresses the specific limitations of module-level analysis in large-scale systems. The authors assert that:

Project-wise similarity is a necessary component for the first stage of large-scale plagiarism detection (retrieval), preceding detailed module-level verification.
The combination of size-based weighting and partial similarity is a complementary mechanism that jointly improves both resilience to code modifications and credibility in distinguishing unrelated projects.
Effective project-wise similarity does not require complex matching strategies; rather, a simple combination of symmetric aggregation, size-aware weights, and selective focus on high-similarity pairs provides a practical and robust solution.
The proposed framework is general and applicable to various types of birthmarks, though the study specifically validates it using static $k$ -gram birthmarks on Java projects.

The authors acknowledge limitations, noting that the dataset consists of open-source Java projects and does not include confirmed real-world plagiarism cases or adversarially obfuscated code. They position their work as a systematic evaluation of similarity aggregation strategies, with future work planned to extend to other languages and investigate robustness against obfuscation.