IntelliSA: An Intelligent Static Analyzer for IaC Security Smell Detection Using Symbolic Rules and Neural Inference

Technical Summary: IntelliSA

Problem Statement
Infrastructure as Code (IaC) enables the automated provisioning of large-scale environments but introduces significant security risks when misconfigurations occur. While static analyzers based on symbolic rules (e.g., SLIC, SLAC, and the state-of-the-art Glitch) have been developed to detect "security smells" (bad coding patterns hinting at vulnerabilities), they suffer from a fundamental limitation: over-approximation. These rule-based tools generate high recall but produce excessive false positives, overwhelming security engineers with manual inspection burdens. Conversely, while Large Language Models (LLMs) can reduce false positives through contextual understanding, relying on them directly via APIs introduces high costs, latency, data governance concerns, and reproducibility issues. Furthermore, standalone LLMs often suffer from under-approximation, missing true positives that rule-based tools catch.

Methodology: IntelliSA Framework
The authors propose IntelliSA, an intelligent static analyzer that integrates symbolic rule matching with neural inference to address the false-positive dilemma. The framework operates in three primary stages:

1. Symbolic Over-Approximation: The system first employs a static analyzer (specifically Glitch) to scan IaC scripts (Puppet, Ansible, Chef) using symbolic rules. This ensures broad coverage and high recall, flagging all potential security smells, including many false positives.
2. Knowledge Distillation for False Positive Filtering: To filter the false positives without relying on expensive LLM APIs during inference, the authors employ a teacher-student knowledge distillation approach:
   • Teacher: A large LLM (Claude-4) acts as a teacher model. It is prompted with the flagged code lines, the specific detection rules, and context to determine if a warning is a True Positive (TP) or a False Positive (FP). These LLM-generated labels serve as pseudo-labels.
   • Student: A compact neural model (CodeT5p-220M, ~220M parameters) is trained using these pseudo-labels. The student learns to classify warnings from the static analyzer as TP or FP. The student model is over 500× smaller than the teacher, enabling efficient, offline deployment.
3. End-to-End Inference: In the final pipeline, the static analyzer generates warnings, which are then passed through the trained student model. Only warnings predicted as True Positives are reported to the user.

Key Contributions

1. IntelliSA Framework: The first intelligent static analyzer for IaC that combines symbolic rules for coverage with distilled neural inference for precision, specifically targeting the reduction of false positives in security smell detection.
2. Cost-Effective Distillation: The demonstration that a compact student model can effectively learn from an LLM teacher's pseudo-labels to filter static analysis warnings, achieving high accuracy without the runtime cost or data privacy risks of direct LLM API usage.
3. Empirical Validation: A comprehensive evaluation against existing static analyzers (SLIC, SLAC, Glitch) and zero-shot LLM baselines (Claude-4, Grok-4, GPT-5) across three major IaC technologies.

Experimental Results
The system was evaluated on a human-labeled dataset comprising 241 IaC scripts (11,814 lines of code) covering 213 security smell instances across Puppet, Ansible, and Chef.

• Accuracy (RQ1): IntelliSA achieved the highest Macro-F1 score of 83%, outperforming the state-of-the-art static analyzer Glitch (69%) and LLM baselines (up to 76%). Specifically, it improved F1 scores by 27% on Puppet, 4% on Ansible, and 11% on Chef compared to Glitch, driven primarily by significant gains in precision (e.g., +43% on Puppet) with only modest reductions in recall.
• Cost-Effectiveness (RQ2): IntelliSA demonstrated superior efficiency in locating smells. It achieved the lowest Effort@60%Recall (the percentage of code lines an engineer must inspect to find 60% of smells), requiring only 1.14% (Puppet), 0.74% (Ansible), and 1.66% (Chef) of the codebase. It also achieved the highest F1@1%LOC (quality of detection within the top 1% of inspected lines), reaching 85% on Ansible.
• Design Choices (RQ3):
  • Teacher Model: Claude-4 provided the best balance for pseudo-labeling (F1 89%), outperforming GPT-5 and Grok-4.
  • Student Model: CodeT5p-220M proved the most effective architecture (Macro-F1 79%).
  • Scale: The 220M parameter model performed comparably to the 770M variant but was 71% smaller, validating the efficiency-oriented design.

Significance and Claims
The paper claims that IntelliSA offers a practical solution to the "false positive" bottleneck in IaC security analysis. By leveraging knowledge distillation, it achieves the high precision of LLMs while retaining the low-cost, offline deployability of traditional static analyzers. The authors assert that IntelliSA is the most accurate and cost-effective approach currently available for detecting security smells in IaC, enabling security engineers to identify the majority of vulnerabilities with minimal manual inspection effort. The work is released as open source to support reproducibility and further research.