Cache-Related Smells in GitLab CI/CD: Comprehensive Catalog, Automated Detection, and Empirical Evidence

1. Problem Statement

Continuous Integration and Deployment (CI/CD) pipelines are critical for rapid software delivery, but poorly configured pipelines lead to increased feedback times, resource consumption, and failure rates. While caching is a well-established technique to improve pipeline speed and reliability, existing research and tools have primarily focused on missing dependency caches (e.g., failing to cache node_modules or pip packages).

This paper identifies a gap: developers often misconfigure other types of caches (artifacts, Docker layers, fallback keys) or fail to utilize advanced caching features (e.g., pull-through caches). These misconfigurations, termed "cache-related smells," negatively impact the speed, efficiency, and reliability of GitLab CI/CD pipelines. There is currently no comprehensive catalog of these smells, nor are there automated tools to detect them beyond simple dependency caching.

2. Methodology

The authors employed a mixed-method approach involving literature mining, tool development, and empirical evaluation.

• Smell Mining & Validation:
  • Source Analysis: The authors analyzed ~150 pages of official GitLab CI/CD documentation and 27 specific sources (blogs, release notes) to identify potential cache-related issues.
  • Validation: They validated these candidate smells against a curated corpus of grey literature (practitioner blogs and articles) to ensure relevance to real-world usage.
  • Result: This process yielded a catalog of 10 distinct cache-related smells.
• Tool Development (CROSSER):
  • The authors developed CROSSER (Cache-Related Observer of Smells for Speed, Efficiency, and Reliability), a static analysis tool written in Kotlin.
  • It parses .gitlab-ci.yml files, resolves inheritance (e.g., extends, variables), and maps values to a meta-model to detect smells.
  • CROSSER is designed to detect 7 out of the 10 identified smells automatically.
• Datasets:
  • D1 (Frequency Study): A dataset of 228 mature open-source GitLab projects (filtered by popularity, activity, and CI/CD usage) to measure smell frequency.
  • D2 (Accuracy Study): A subset of 82 manually labeled projects from D1, selected based on maturity metrics, to evaluate CROSSER's detection accuracy.

3. Key Contributions

A. Comprehensive Catalog of 10 Cache-Related Smells

The paper defines ten specific anti-patterns, categorized by their impact on Speed (SP), Efficiency (EF), and Reliability (RE):

1. Artifacts Default Expiration Period: Failing to set expire_in, leading to 30-day retention of unnecessary files (Misconfiguration).
2. Artifacts Fetched by Default: Jobs fetching artifacts they do not need because explicit dependencies (dependencies/needs) were not defined (Misconfiguration).
3. No Dependencies Cache: Failing to cache dependency installation directories (e.g., .gradle, node_modules) (Missing Cache).
4. No Fallback Cache: Failing to define fallback_keys, causing pipelines to start "cold" when branch-specific caches are missing (Missing Cache).
5. God Cache: Storing unrelated files (e.g., build artifacts and test coverage) under a single cache key, forcing unnecessary downloads (Misconfiguration).
6. Use of a Protected Cache: Unnecessarily using protected branch caches for unprotected branches, causing cache misses due to permission restrictions (Misconfiguration).
7. Redundant Cache Updates: Multiple jobs pushing to the same cache key simultaneously, causing race conditions and slowdowns (Misconfiguration).
8. Suboptimal Compression Type: Using default compression settings that are not optimal for specific file types or network conditions (Misconfiguration).
9. No Docker Pull-Through Cache: Pulling Docker images directly from Docker Hub instead of using GitLab's Dependency Proxy (Missing Cache).
10. No Docker Layers Cache: Building Docker images without reusing previous layers (missing --cache-from), forcing full rebuilds (Missing Cache).

Note: Three smells (God Cache, Protected Cache, Suboptimal Compression) were deemed too complex or context-dependent for automatic detection in the current version of CROSSER.

B. CROSSER Tool

• Capabilities: Automatically detects 7 of the 10 smells.
• Performance: Achieved an overall F1 score of 0.98 on the manually labeled dataset (D2).
• Features: Handles complex YAML features like global defaults, variables, and job inheritance.

C. Empirical Evidence

The authors conducted a large-scale study on 228 mature projects to quantify the prevalence of these issues.

4. Results

• Detection Accuracy (RQ2):
  • CROSSER achieved an aggregate F1 score of 0.98.
  • Four detectors achieved perfect precision and recall (1.00).
  • Errors were primarily due to non-standard commands (e.g., docker buildx) or complex variable nesting that the parser could not resolve.
• Frequency of Smells (RQ3):
  • Widespread Prevalence: Only 11% of the 228 projects were completely smell-free.
  • Median: The median number of smells per repository was 3.
  • Specific Findings:
    • No Docker Pull-Through Cache was the most common, affecting 85.9% of group repositories.
    • Artifacts Default Expiration Period and Artifacts Fetched by Default affected over 50% of repositories.
    • No Fallback Cache affected 42.1% of repositories.
  • Impact: Over 80% of pipelines were impacted in terms of Speed, Efficiency, or Reliability.
• Developer Awareness:
  • The study suggests developers are often unaware of advanced features. For instance, the "Pull-Through Cache" (available since 2020) and "Fallback Keys" (available since 2023) are rarely used despite their significant performance benefits.

5. Significance and Implications

• Beyond Dependencies: This work shifts the focus from the well-studied "missing dependency cache" to a broader spectrum of cache misconfigurations that are equally detrimental to pipeline performance.
• Actionable Tooling: CROSSER provides practitioners with an immediate, automated way to audit their CI/CD configurations, potentially saving significant build time and reducing failure rates.
• Documentation Gaps: The findings highlight that official CI/CD documentation is often fragmented. Critical best practices are scattered across different pages, making them difficult for developers to internalize. The authors call for more quantitative examples and unified guidelines in platform documentation.
• Future Directions: The paper suggests that while static analyzers (like CROSSER) are effective, future work could explore Large Language Models (LLMs) to detect more complex smells (like "God Cache" or "Suboptimal Compression") that require semantic understanding of file contents and hardware context.

In conclusion, the paper establishes that cache-related anti-patterns are ubiquitous in mature software projects and provides both a theoretical catalog and a practical tool (CROSSER) to address them, significantly contributing to the optimization of CI/CD workflows.