🤖 AI

LSem2Vec: A Simple yet Effective Two-Stage Approach for Source Code Embedding

This paper introduces LSem2Vec, a simple yet effective two-stage framework that combines large language models for semantic extraction with sentence embedding models to generate robust source code representations without requiring costly task-specific training or fine-tuning, outperforming existing unsupervised methods across multiple datasets.

Original authors: Zixiang Xian, Chenhui Cui, Rubing Huang, Chunrong Fang, Zhenyu Chen

Published 2026-08-19

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Zixiang Xian, Chenhui Cui, Rubing Huang, Chunrong Fang, Zhenyu Chen

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

In the vast landscape of modern software, lines of code are the bricks and mortar of our digital world. Just as a city planner needs to understand the layout of streets and buildings to manage a metropolis, software engineers need to understand the structure and meaning of code to maintain, improve, and secure the systems they build. A critical challenge in this field is recognizing when two pieces of code are essentially doing the same thing, even if they look different on the surface. This is known as finding "clones," and it helps developers avoid redundancy and spot security risks. For years, computers have struggled with this task because they often get lost in the sheer volume of text or fail to grasp the underlying logic when the wording changes. While powerful artificial intelligence tools have recently emerged that can read and write code, using them to compare thousands of files has proven difficult, expensive, and prone to mistakes, often because the tools get overwhelmed by the length of the code or give incorrect answers when asked to make complex judgments all at once.

A team of researchers has now introduced a new method called LSEM2VEC that solves these problems by changing how the computer "reads" the code. Instead of asking a massive artificial intelligence to stare at two long files and decide if they are similar—a task that often leads to confusion or errors—the new approach breaks the job into two simple, manageable steps. First, the system uses a large language model to act as a translator, reading a chunk of code and writing a single, clear sentence that summarizes what that code does. This step strips away the confusing details and leaves only the core meaning. Then, a second, specialized tool takes that summary sentence and converts it into a mathematical point in space, known as an embedding. By turning the code into these points, the computer can easily measure the distance between them to see how similar they are, without ever needing to re-read the original, lengthy files. This process is like having a librarian who first writes a one-sentence description of every book in a massive library and then groups the books based on those descriptions, rather than trying to read every page of every book to find matches.

The researchers tested this method on three different sets of code written in various programming languages, including C and Java, using several different artificial intelligence models to ensure the results were robust. They compared their new approach against many existing methods, including those that require extensive training on labeled data or those that try to use artificial intelligence directly for the comparison. The results were striking: the new method consistently outperformed the others, finding code clones with much higher accuracy. In one test involving C code, the system achieved an accuracy score of over 95 percent, significantly beating the next best method. It also proved highly effective at grouping similar code together, a task known as clustering, where it achieved a score of 0.99 on the Adjusted Rand Index, surpassing even methods that had been trained with human supervision, which achieved a score of 0.90.

A key advantage of this work is that it does not require the expensive and time-consuming process of training the artificial intelligence on specific datasets. Traditional methods often need thousands of examples of code pairs labeled by humans to learn how to spot similarities, which is slow and costly. The new approach works immediately, using the existing knowledge of the artificial intelligence models without any extra training. It also solves a major technical hurdle: the limited memory of these models. Large language models can only process a certain amount of text at once; if the code is too long, the model crashes or gives up. By summarizing the code first, the researchers bypassed this limit, allowing the system to handle large files that would have previously been impossible to analyze. Furthermore, the method is much more efficient, requiring far fewer calls to the artificial intelligence models, which saves both time and money.

The study also explored how different choices affect the outcome, such as using different types of artificial intelligence models or removing common "stop words" from the summaries. They found that while the specific tools matter, the overall approach remains strong across different configurations. For instance, using a more advanced artificial intelligence model to write the summaries led to better results, but even the standard models performed exceptionally well. The researchers also visualized the results, showing that the code points generated by their method formed tight, clear groups, whereas other methods produced messy, overlapping clusters. This clarity suggests that the system truly understands the meaning of the code rather than just matching surface-level patterns.

Ultimately, this research offers a practical and efficient way to understand the vast oceans of code that power our world. By simplifying the complex task of code comparison into a two-step process of summarization and measurement, the researchers have created a tool that is both powerful and accessible. It demonstrates that we do not always need to build bigger, more complex models to solve difficult problems; sometimes, a smarter way of using the tools we already have is enough to see the forest for the trees. This approach could help software engineers clean up their codebases, find hidden security vulnerabilities, and organize their projects more effectively, all without the heavy computational cost that has previously limited these capabilities.

Technical Summary of LSEM2VEC: An Effective Embedding Approach for Code Clone Detection and Clustering

Problem Statement
The integration of Large Language Models (LLMs) into software engineering (SE) has advanced tasks like source code clone detection and clustering. However, existing methods for generating source code embeddings face significant limitations. Traditional pre-trained models (PTMs) often require extensive supervised training or fine-tuning on task-specific data, demanding substantial computational resources and labeled datasets. While LLMs possess impressive zero-shot learning capabilities, directly applying them to SE tasks presents two critical challenges:

Erroneous Responses: LLMs frequently produce incorrect outputs when tasked with complex reasoning, such as determining if two code snippets are clones, especially when the workflow involves summarizing functionality and then making a binary decision.
Context Length Limitations: LLMs have fixed context windows. Processing large volumes of source code or comparing multiple code pairs simultaneously often exceeds these limits, leading to errors or the inability to process long code fragments. Furthermore, direct pairwise comparison using LLMs results in quadratic time complexity ( $O(n^2)$ ) regarding LLM calls, making it inefficient and costly for large datasets.

Methodology: LSEM2VEC
The authors propose LSEM2VEC (LLM-extracted code Semantics to Vector embedding), a novel, training-free approach that generates source code embeddings by combining LLMs with sentence embedding models. The framework operates via a four-stage pipeline designed to mitigate the aforementioned challenges through a "late-interaction" strategy:

Uniform Prompt Design: A standardized prompt template instructs the LLM to read a source code fragment and succinctly summarize its function in a single sentence, explicitly forbidding explanatory text. This ensures compatibility with sentence embedding models, which expect single-sentence inputs.
Code Summaries and Storage: The LLM generates the one-sentence summary for each code fragment. Crucially, the system extracts only the first sentence of the response to ensure consistency and stores these summaries in JSON format. This step decouples the semantic extraction from the downstream task, allowing summaries to be reused without re-querying the LLM.
Code Embedding Generation: The stored text summaries are fed into a pre-trained sentence embedding model (e.g., MiniLM or SBERT-CN) to generate fixed-dimensional vector representations. This stage converts the natural language summaries into semantic vectors without any additional training or fine-tuning.
Downstream Tasks: The resulting embeddings are used for unsupervised tasks. For code clone detection, similarity is calculated using cosine similarity between vector pairs. For code clustering, Euclidean distance is used with the K-means algorithm.

This "late-interaction" design reduces the computational complexity of code clone detection from $O(n^2)$ (requiring an LLM call for every code pair) to $O(n)$ (requiring only one LLM call per code fragment), significantly lowering token costs and avoiding context length overflow.

Key Contributions
The paper claims the following contributions:

First Training-Free Integration: It is the first work to employ large language and sentence embedding models to generate source code embeddings without requiring any training or fine-tuning, utilizing unlabeled datasets.
Mitigation of LLM Limitations: The proposed pipeline effectively addresses erroneous LLM responses and context-length constraints by breaking the task into smaller steps (summarization followed by embedding) rather than asking the LLM to perform the entire reasoning task directly.
Flexibility and Scalability: The approach is LLM-independent, allowing the use of any open-source or proprietary LLM and various sentence embedding models. It scales across different programming languages without needing language-specific training data.
Efficiency: By reducing the number of LLM calls and leveraging pre-trained sentence encoders, the method offers a resource-efficient solution compared to fine-tuned models or direct LLM prompting.

Experimental Results
The authors evaluated LSEM2VEC on three datasets (POJ-104, OJClone C, and BigCloneBench) covering C and Java languages. The evaluation focused on unsupervised code clone detection and code clustering using GPT-3.5, GLM3, and GLM4.

Code Clone Detection: LSEM2VEC significantly outperformed state-of-the-art unsupervised methods, including SourcererCC, Code2vec, InferCode, TransformCode, and LLM2Vec.
- On the OJClone C dataset, the best configuration (GPT-3.5 + MiniLM-L12) achieved an F1-score of 91.82%, surpassing the supervised LLM2Vec (83.70%) and other unsupervised baselines.
- On the BigCloneBench dataset, LSEM2VEC achieved F1-scores up to 85.83% (using GPT-3.5 + MiniLM-L6), again outperforming the supervised LLM2Vec and other methods.
- The study found that GPT-3.5 generally yielded better results than the GLM series models (GLM3 and GLM4), and the choice of sentence embedding model (e.g., MiniLM-L12 vs. L6) impacted performance.
Code Clustering: Using the Adjusted Rand Index (ARI) as a metric, LSEM2VEC demonstrated superior clustering capabilities.
- The GPT-3.5 configuration achieved an ARI of 0.97 (using MiniLM-L6) and 0.91 (using MiniLM-L12), outperforming the supervised LLM2Vec (0.90) and all other unsupervised baselines.
- Visualization via t-SNE confirmed that LSEM2VEC embeddings produced well-defined, distinct cluster boundaries, particularly when using more advanced LLMs like GPT-3.5 compared to GLM3.

Significance and Claims
The paper positions LSEM2VEC as a robust and efficient alternative to existing embedding strategies. Its primary significance lies in demonstrating that high-quality source code embeddings can be generated without the computational overhead of fine-tuning or the dependency on labeled data. By leveraging the zero-shot capabilities of LLMs for semantic extraction and sentence models for vectorization, the approach provides a practical solution for real-world SE applications where resources are limited. The authors conclude that this method offers a new direction for the community to explore advanced source code representations, potentially extending to tasks like code summarization, vulnerability detection, and binary code analysis in future work.

More like this