An LLM-based Technique for Detection of Code Clones Across Open Source Microservice Repository Forks

In the vast, collaborative world of software development, programmers often build new applications by taking existing code and adapting it for a different purpose. This practice, known as forking, is the engine of open-source innovation, allowing teams to stand on the shoulders of giants rather than starting from scratch. However, as these projects grow and branch out, a hidden complexity arises: the code within them can become a tangled web of duplicates. Some copies are exact, line-for-line replicas, while others are cleverly rewritten versions that do the same thing but look completely different on the surface. These are known as code clones. For the maintainers of large software systems, especially those built as a collection of small, independent services, finding these duplicates is a monumental task. Traditional tools are excellent at spotting exact copies, but they often fail to recognize when two pieces of code have been refactored to perform the same function in entirely different ways. Understanding where and how these duplicates exist is crucial, as it reveals how software evolves, where risks of inconsistency lie, and how much effort is truly required to keep a system running smoothly.

A team of researchers recently set out to solve this problem by testing a new kind of digital tool: large language models. These are the same powerful artificial intelligence systems that can write stories or answer questions, but here they were trained to understand the logic of computer code. The researchers wanted to see if these models could act as a sophisticated microscope, spotting not just identical code, but also the semantically similar code that traditional tools miss. They focused their investigation on open-source microservice projects, which are modern software systems built from many small, interchangeable parts. To begin, they needed to teach these models how to recognize the different types of code clones. They gathered a massive dataset of ten thousand pairs of C++ code snippets, half of which were known to be functionally identical but syntactically different, and the other half were unrelated. They tested three different artificial intelligence models, ranging from a general-purpose system to two specialized models designed specifically for coding tasks. They also experimented with different ways of asking the models to perform the task, trying everything from simple instructions to complex prompts that provided examples and asked for confidence scores.

The results of this initial testing were revealing. The researchers found that the specialized coding models were far superior to the general-purpose one. Among them, a model named Codestral emerged as the most accurate, achieving the highest overall accuracy of 65.5% and outperforming both the other specialized model and the general-purpose system. However, the study also uncovered a significant limitation: even the best model made mistakes. It frequently missed the very duplicates it was supposed to find, failing to recognize them about forty percent of the time. While it was good at identifying code that was definitely different, it was too cautious when it came to spotting the subtle similarities. This high rate of missed detections meant that, for now, these powerful tools are not yet reliable enough to be used as the sole method for managing code in a professional, automated setting. They are useful assistants, but they cannot yet replace human judgment or established detection methods.

Armed with the best-performing model and the most effective way of prompting it, the researchers then turned their attention to the real world. They selected twenty-two pairs of open-source microservice repositories, where one was the original project and the other was a fork that had been modified by a different group of developers. They fed these codebases into their system to map out exactly what kinds of duplicates existed between the original and the fork. The findings painted a clear picture of how these projects evolve. The most common type of duplicate they found was the exact copy, where the code remained unchanged. The second most common was the Type-3 clone, where the code had been altered with additions or deletions but retained its original functionality. Surprisingly, the completely rewritten code that does the same thing in a totally different way was less common than the other types in these specific projects. This suggests that when developers fork a microservice, they tend to keep the core structure intact rather than completely reinventing the wheel.

To ensure their automated findings were trustworthy, the researchers compared the model's output against a manual review of three of the projects by human experts. The human reviewers confirmed that the model was generally correct in identifying the most common types of duplicates, though the specific numbers varied depending on which underlying technology the model used to read the code. The study concluded that while artificial intelligence offers a promising new way to understand the deep, semantic relationships between pieces of code, it is not yet a perfect solution. The technology is powerful enough to see patterns that humans and older tools might miss, but it still struggles with the high stakes of missing a duplicate entirely. For now, these models serve best as a guide to help developers navigate the complex history of their code, rather than as a final authority. The path forward involves refining these tools to reduce their errors and combining their semantic understanding with traditional methods to create a more complete picture of software evolution.