💻 computer science

Not All Tokens Matter: Data-Centric Optimization for Efficient Code Summarization

This paper demonstrates that efficient code summarization requires language-specific token curation strategies rather than uniform reduction, revealing that Abstract Syntax Trees significantly boost performance in Java while Function Signatures are optimal for Python, challenging the assumption of cross-language transferability in data-centric optimization.

Original authors: Saima Afrin, Zaiyu Cheng, Tushar Sharma, Alexander Serebrenik, Massimiliano Di Penta, Antonio Mastropaolo

Published 2026-07-20

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Saima Afrin, Zaiyu Cheng, Tushar Sharma, Alexander Serebrenik, Massimiliano Di Penta, Antonio Mastropaolo

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to teach a super-smart robot how to read a library of millions of books and write a short summary for each one. This is the world of Large Language Models (LLMs), the AI engines behind tools that can write code, answer questions, and even tell jokes. But here's the catch: these robots are incredibly hungry. To learn, they need to chew through massive amounts of text, which requires giant, expensive computers and a lot of electricity. It's like trying to feed a dragon a whole forest just to get it to say "hello."

In the world of software, this robot's job is code summarization: taking a block of computer instructions (code) and turning it into a simple sentence that explains what it does. Think of it as translating a complex recipe into a quick headline like "Make a cake." The problem is that computer code is often full of repetitive, boring words and symbols that don't actually change the meaning of the recipe. If you try to teach the robot by feeding it every single word, including the "and," "the," and "if," you waste time and energy. The big question researchers have been asking is: Can we teach the robot just the important parts, skip the fluff, and still get a perfect summary?

This paper, titled "Not All Tokens Matter," dives right into that question. The authors, a team of computer scientists, decided to test a bold idea: what if we don't need to feed the robot the whole code? What if we can cut out the boring parts before the robot even sees them? They didn't just guess; they ran three different experiments to see which way of "trimming the fat" works best. They tried turning code into a structural diagram, stripping it down to just the function names, and using a smart filter to remove common, useless words.

Here is the twist they discovered, and it's a bit like a magic trick that only works on certain days. They found that there is no single "best" way to trim the code. It depends entirely on the language the code is written in.

When they worked with Java (a language that is very strict and uses a lot of words to say simple things), the best strategy was to turn the code into a structural diagram (called an Abstract Syntax Tree). This method chopped out about 56% to 73% of the words but actually made the robot better at summarizing, improving its score by 37%. It was like taking a dense, wordy novel and turning it into a clear, organized outline—the robot understood it perfectly.

However, when they switched to Python (a language known for being short and sweet), that same structural diagram was a disaster. It made the robot's performance drop by nearly 50%. Why? Because Python relies heavily on the specific names of things to make sense. When they tried the diagram, they accidentally threw away the very clues the robot needed. Instead, for Python, the winning strategy was to throw away almost everything except the Function Signatures (the title and the ingredients list of the code). This method removed a massive 83% of the tokens but kept the quality high. It turned out that for Python, the "title" tells you everything you need to know.

There was also a third method, called CrystalBLEU, which acted like a smart eraser, removing common words that appear everywhere but don't add meaning. This one was a reliable "middle ground," working well for both languages and cutting out about 60% to 72% of the text without hurting the results too much.

The team also built a new, high-quality test set of Python code summaries called PyBench to make sure their results were real, and they created a new tool called SIDEpy to check if the code and the summary actually matched up in meaning, not just in words. They found that simply removing words isn't enough; you have to remove the right words. If you cut out the wrong ones, the robot gets confused.

In the end, the paper suggests that the old idea of "bigger is better" is wrong. You don't need to feed the robot the whole forest. If you know the language, you can give it a carefully curated handful of leaves, and it will learn just as well, if not better. The key takeaway is that one size does not fit all: what works for Java will break Python, and vice versa. To make AI efficient, we need to be careful editors, not just big feeders, tailoring our approach to the specific style of the code we are teaching.

Technical Summary: Not All Tokens Matter: Data-Centric Optimization for Efficient Code Summarization

Problem Statement
The rapid advancement of Large Language Models (LLMs) has revolutionized automated code summarization, yet training these models remains computationally expensive, particularly when handling millions of code-comment pairs. A significant challenge is the "token bloat" inherent in raw source code, where syntactically necessary but semantically uninformative tokens (e.g., delimiters, boilerplate, common keywords) dilute meaningful training signals and inflate computational costs. Furthermore, existing datasets often contain noisy or misaligned $\langle code, comment \rangle$ pairs, which can hinder generalization. While prior work has explored dataset-level filtering (e.g., removing low-quality pairs), there is a lack of understanding regarding how finer-grained, token-level optimization affects model efficiency and summary quality across different programming languages.

Methodology
The authors propose a data-centric optimization framework that investigates strategic token reduction to minimize computational overhead while maintaining or improving summary quality. The study employs three distinct token-level reduction techniques, evaluated on Java and Python datasets (Funcom) using the CodeT5+ (220M) model:

AST-based Optimization: Replaces raw source code with Abstract Syntax Tree (AST) representations, stripping superficial syntactic tokens (delimiters, formatting) while retaining structural fidelity.
Function Signature (FS) Optimization: Retains only the function definition (name, return type, parameters), discarding the entire function body.
CrystalBLEU-guided Pruning: Adapts the CrystalBLEU metric to identify and remove "trivially shared" $n$ -grams (frequent, low-discrimination tokens like public or return) from the input sequence.

These strategies are evaluated in two configurations:

Standalone: Applied directly to the cleaned dataset.
Cascaded: Applied after semantic filtering using the SIDE (Summary alIgnment to coDe sEmantics) metric, which removes misaligned $\langle code, comment \rangle$ pairs.

To address the lack of language-specific semantic alignment metrics for Python, the authors developed $SIDE_{py}$ , a retrained contrastive model specifically for Python, and created PyBench, a new high-quality benchmark of 500 manually curated Python code-summary pairs.

Key Contributions

Empirical Evaluation of Token Reduction: A comprehensive study demonstrating that token-level optimization strategies are highly language-dependent. The paper challenges the assumption that a single reduction strategy is universally optimal.
Development of $SIDE_{py}$ : A Python-specific semantic alignment metric that shows a 36.5% improvement in correlation with human adequacy judgments compared to applying the Java-trained SIDE metric "out-of-the-box" to Python.
PyBench Benchmark: A new, rigorously curated Python evaluation dataset (500 instances) with human annotations across content adequacy, fluency, and conciseness, designed to mitigate data contamination from pre-training corpora.
Replication Package: A complete open-source package containing datasets, models, benchmarks, and analysis scripts to support reproducible research.

Results
The study reveals stark contrasts in how different languages respond to token reduction:

Java: AST-based optimization yields the best results, achieving a 37% performance improvement (BLEU) over the baseline while reducing tokens by 56–73%. Function Signatures perform poorly in Java, causing significant degradation.
Python: The pattern inverts. Function Signature-based optimization is optimal, achieving 83% token reduction while maintaining or slightly improving quality (e.g., 140% of baseline BLEU on PyBench). Conversely, AST-based optimization causes catastrophic performance degradation (up to 49% loss) in Python.
CrystalBLEU: Demonstrates cross-language robustness, achieving 60–72% token reduction with moderate performance trade-offs, making it a reliable fallback for both languages.
Entropy vs. Token Count: The authors emphasize that Shannon entropy reduction is a more principled measure of compression than raw token count. Optimized inputs not only shorten sequences but also reduce information-theoretic complexity, allowing models to converge faster.
Cascading Effects: Semantic filtering (SIDE/ $SIDE_{py}$ ) enables more aggressive token compression without quality loss. For instance, in Java, applying AST reduction after semantic filtering maintained performance while reducing tokens significantly.

Significance and Claims
The paper argues that "which tokens are kept matters more than how many are removed." The primary significance lies in demonstrating that data-centric optimization cannot be a "one-size-fits-all" approach; strategies must be calibrated to the specific syntactic and semantic characteristics of the programming language.

Language-Specificity: The findings challenge the transferability of optimization strategies across languages. Python's concise syntax and descriptive naming conventions make Function Signatures highly informative, whereas Java's verbose, structurally explicit syntax relies on AST representations to preserve semantic context.
Efficiency and Quality: Strategic token reduction can enhance model efficiency (reducing training time and energy) without sacrificing, and in some cases improving, summary quality by filtering out noise and focusing the model on high-signal tokens.
Data-Centric Paradigm: The work reinforces the shift from "more data is better" to "quality data is better," showing that combining instance-level filtering (removing bad pairs) with token-level optimization (removing bad tokens) creates a synergistic effect for efficient code summarization.

The authors conclude that future work in code summarization must adopt language-aware evaluation frameworks and that practitioners should validate token optimization strategies per language before deployment.

More like this