Technical Summary: Genetic Programming for Self-Adaptive Auto-Scaling of Microservices

Problem Statement

Microservice architectures offer scalability and flexibility but introduce significant complexity in auto-scaling. Traditional approaches often rely on reactive resource allocation based on individual metrics (e.g., CPU or memory usage) or fixed thresholds. These methods frequently fail to balance two competing goals: satisfying Service-Level Objectives (SLOs) and minimizing operational costs.

• Under-provisioning: Insufficient resources lead to SLO violations (e.g., latency spikes or request failures), degrading service quality.
• Over-provisioning: Excessive resource allocation to avoid violations results in unnecessary costs.
  Existing solutions, including Kubernetes' default Horizontal Pod Autoscaler (HPA), often lack system-level awareness, react only after violations occur, or fail to learn from past adaptations, leading to repeated "one-off" scaling decisions rather than evolving robust policies.

Methodology: AutoSLO

The authors propose AutoSLO, a learning-based, self-adaptive framework that uses Genetic Programming (GP) to evolve reusable scaling logic. Instead of generating isolated adjustments at runtime, AutoSLO learns mathematical formulas that map runtime operational metrics to the optimal number of microservice replicas (pods).

Core Components

1. Prerequisite Artifacts (Offline):

   • Bottleneck Microservice Identification: Using Performance Bottleneck Analysis (PBA), AutoSLO identifies a small subset of services that degrade end-to-end performance under load. Scaling is focused exclusively on these bottlenecks to reduce the search space.
   • Surrogate Model: A regression-based model (Random Forest Regressor) is trained offline to predict SLO outcomes (e.g., 90th-percentile latency or success rate) for candidate scaling strategies. This avoids the high cost of deploying and testing every candidate strategy online.

2. Self-Adaptation Control Loop (Online):
   The system operates in a continuous four-step loop:

   • Monitoring: Collects operational metrics (CPU, memory, QPS) and SLO metrics via Prometheus and Istio.
   • SLO Checking: Determines if the system is under-provisioned (SLO violated) or over-provisioned (resources excessive). If the status is stable, the system continues with the last known best strategy.
   • Planning (GPPlan): If adaptation is needed, the GP algorithm evolves a population of scaling formulas.
     • Representation: Each individual is a list of mathematical expressions (one per bottleneck service) combining metrics and constants (e.g., qps/5 + cpu).
     • Fitness Evaluation: The surrogate model predicts the SLO outcome for each candidate. Fitness is minimized based on:
       • SLO Violation Severity: If the predicted metric exceeds the threshold, fitness penalizes the deviation.
       • Resource Efficiency: If the SLO is met, fitness rewards minimizing the total number of pods.
     • Evolution: Standard GP operators (crossover, mutation, tournament selection) are used to evolve better formulas over generations.
   • Updating: The best evolved formula (bestListF) is applied to the Kubernetes cluster to adjust pod counts.

Key Contributions

• Systematic Application of Search-Based Software Engineering (SBSE): The paper applies Genetic Programming to the specific problem of online auto-scaling, moving beyond static heuristics to learn dynamic, reusable policies.
• Proactive vs. Reactive: Unlike reactive systems that search for a solution only after a violation, AutoSLO evolves strategies that can proactively prevent violations by generalizing across changing workloads.
• Bottleneck-Aware Optimization: By focusing scaling efforts only on identified bottleneck services, the framework reduces computational overhead and search complexity.
• Evaluation on Diverse Architectures: The framework is validated on both CPU-intensive (e-commerce) and GPU-intensive (LLM-based chatbot) systems, demonstrating generalizability across different resource constraints.

Experimental Results

The authors evaluated AutoSLO against two baselines: the industry-standard Kubernetes HPA and a Random Search variant of AutoSLO (AutoSLO-Ran).

• Resource Efficiency:
  • Boutique Shop (CPU-intensive): AutoSLO reduced average pod usage by 50.65% compared to HPA (12.36 pods vs. 25.05 pods).
  • Chatbot (GPU-intensive): AutoSLO reduced average pod usage by 46.4% compared to HPA (1.57 pods vs. 2.93 pods).
  • Compared to AutoSLO-Ran, AutoSLO achieved a statistically significant reduction in pod usage for the GPU-intensive system.
• SLO Violations:
  • HPA avoided violations entirely but at the cost of significant over-provisioning.
  • AutoSLO incurred a low frequency of SLO violations (average < 2.8 per run) but resolved them rapidly (within < 2 minutes, or two control-loop cycles).
  • The authors frame these brief violations as a deliberate trade-off to achieve resource efficiency, contrasting with HPA's conservative over-provisioning.
• Statistical Significance: Using the Wilcoxon Rank-Sum test and Vargha–Delaney effect size, the reduction in resource usage was statistically significant (p < 0.01) compared to HPA.

Significance and Claims

The paper claims that AutoSLO addresses a fundamental trade-off in microservice management: balancing service quality against operational cost. Its significance lies in:

1. Shift from Reactive to Adaptive: Moving away from fixed thresholds and one-off adjustments toward evolving, reusable scaling logic that improves over time.
2. Applicability to Emerging Systems: Demonstrating that SBSE techniques can effectively manage complex, modern systems, including those relying on Large Language Models (LLMs) and scarce GPU resources.
3. Practical Viability: Showing that GP can run efficiently online when combined with surrogate modeling and bottleneck identification, making it feasible for real-world deployment without prohibitive overhead.

The authors acknowledge limitations, noting that the current approach assumes a fixed set of bottleneck services and handles a single SLO at a time, though they suggest these could be extended in future work.