Beyond Strict Rules: Assessing the Effectiveness of Large Language Models for Code Smell Detection

Technical Summary: Beyond Strict Rules: Assessing the Effectiveness of Large Language Models for Code Smell Detection

Problem Statement
Code smells are indicators of potential quality degradation that negatively impact software maintainability, reliability, and development costs. While static analysis tools (e.g., JDeodorant, PMD) have long been used for automated detection, they rely on rigid rules that often lack the flexibility to handle the nuanced, context-dependent nature of many smells. Conversely, manual inspection is accurate but expensive and slow. Although Large Language Models (LLMs) have shown promise in various software engineering tasks, their effectiveness in detecting code smells remains underexplored, particularly regarding their ability to outperform or complement traditional tools in real-world, complex codebases. Furthermore, existing benchmarks often rely on small, synthetic datasets or lack robust human-validated ground truth.

Methodology
The authors conducted an empirical study evaluating the effectiveness of four LLMs—DeepSeek-R1, GPT-5 mini, Llama-3.3, and Qwen2.5-Code—in detecting nine specific code smells (Data Class, Dispersed Coupling, Feature Envy, Intensive Coupling, Long Method, Large Class, Long Parameter List, Refused Bequest, and Shotgun Surgery).

• Dataset Construction: The study utilized an extended dataset of 30 top-starred, real-world Java projects from GitHub, comprising over 3.5 million lines of code. To establish a reliable ground truth, the authors sampled 268 code-smell candidates and engaged 76 undergraduate Computer Science students (with verified Java proficiency) to manually evaluate them. A "smell" was confirmed if the average human rating exceeded 3 out of 5.
• Experimental Setup: The LLMs were prompted using a Chain-of-Thought (CoT) strategy, where models were asked to answer specific, literature-based diagnostic questions before concluding whether a smell was present. The temperature was standardized across models to ensure consistency.
• Evaluation Strategy: The study employed three analytical approaches:
  1. Individual LLM Performance: Comparing each LLM's Precision, Recall, and F1-Score against the human ground truth.
  2. LLM vs. Static Analysis: Benchmarking LLMs against four established static analysis tools (JDeodorant, JSpIRIT, Organic, PMD).
  3. Combined Strategy: Proposing and evaluating a voting-based ensemble method that aggregates outputs from the four LLMs and two static analysis tools (6 votes total), flagging a smell if it receives at least three votes.

Key Contributions

1. Ground Truth Dataset: Creation of a manually validated ground truth for 268 code-smell instances across 30 real-world Java projects, serving as a benchmark for future detection strategies.
2. Comprehensive Empirical Evaluation: A systematic comparison of four diverse LLMs against traditional static analysis tools across nine distinct code smell types.
3. Hybrid Detection Strategy: The proposal and validation of a combined voting strategy that integrates LLMs and static tools to improve detection coverage.
4. Replication Package: Provision of all datasets, scripts, and prompt templates to facilitate replication and extension by the research community.

Results
The study reveals that the effectiveness of detection strategies is highly dependent on the specific type of code smell:

• Structurally Simple Smells: For smells defined by clear structural metrics (e.g., Large Class, Long Method, Data Class), LLMs performed strongly, often matching or exceeding static analysis tools. The combined voting strategy achieved the highest F1-scores and Recall for these categories (e.g., Large Class F1: 0.93, Long Method F1: 0.89).
• Context-Dependent and Subjective Smells: For smells requiring deep semantic understanding or context (e.g., Feature Envy, Refused Bequest, Long Parameter List), performance varied significantly. No single strategy dominated all smells. In these cases, specialized tools (e.g., Organic for Long Parameter List) or specific LLMs (e.g., Llama-3.3 for Feature Envy) often outperformed the combined ensemble.
• Trade-offs: The combined strategy consistently maximized Recall, making it effective for minimizing false negatives. However, this came at the cost of increased False Positives for complex smells, suggesting that the optimal strategy depends on whether a project prioritizes Recall (coverage) or Precision (accuracy).
• Model Variability: Performance varied across LLMs; for instance, Llama-3.3 and DeepSeek-R1 generally outperformed others in specific categories, indicating that model architecture and training data influence detection capabilities.

Significance and Claims
The authors claim that while LLMs are not a universal replacement for static analysis tools, they represent a powerful, flexible complement. The study concludes that:

• LLMs are particularly effective for detecting smells based on syntactic patterns and size.
• A "one-size-fits-all" approach is insufficient; the choice of detection tool should be tailored to the specific smell type and the project's tolerance for false positives versus false negatives.
• A combined strategy offers a robust alternative for practitioners seeking high coverage, especially when the optimal individual tool is unknown, though it may require more manual review effort for complex smells.
• The provided dataset and findings offer a foundation for refining detection workflows, improving prompt engineering, and developing hybrid detection systems that leverage the strengths of both rule-based and semantic-based approaches.