🤖 AI

CLOUDADV: Decision-Aligned Instance Sizing with Zero-Shot Foundation Models under Drift

The paper introduces CLOUDADV, an interactive advisory system that leverages zero-shot foundation models to generate cost-effective, drift-resilient cloud instance sizing recommendations, demonstrating significant cost savings and low exceedance rates in production environments without the need for per-tenant model retraining.

Original authors: Jack Bell, Giacomo Carfi, Gerlando Gramaglia, Andrea Simioni, Daniele Fontani, Vincenzo Lomonaco

Published 2026-07-01

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Jack Bell, Giacomo Carfi, Gerlando Gramaglia, Andrea Simioni, Daniele Fontani, Vincenzo Lomonaco

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you own a fleet of delivery trucks. For years, you've been renting the biggest, most expensive trucks possible because you're afraid of running out of space on a busy day. But here's the problem: 90% of the time, these massive trucks are driving around half-empty. You're paying a fortune for gas and rental fees you don't need, while the trucks sit idle.

This is exactly what happens in the cloud. Companies rent powerful virtual computers (VMs) to run their software, but they often "over-provision," meaning they rent way more power than they actually use. This wastes money and energy.

Enter CLOUDADV, a new "smart advisor" system designed to help engineers figure out the right size for their cloud computers without wasting a penny. Here is how it works, broken down into simple concepts:

1. The Problem: The "Crystal Ball" Dilemma

To fix the waste, you need to predict the future. You need to know: Will my computer need more power next week? Or can I downsize?
Traditionally, to make these predictions, you had to build a custom "crystal ball" (a machine learning model) for every single computer. If the computer's behavior changed (like a sudden spike in traffic), that crystal ball would break, and you'd have to rebuild it from scratch. It was slow, expensive, and a lot of work.

2. The Solution: The "Universal Translator"

The researchers used a new type of AI called a Zero-Shot Foundation Model (specifically, a model named Chronos-2).

The Analogy: Think of a traditional model as a specialist who only speaks one language (e.g., "I only understand traffic data for this specific truck"). If you give them a different truck, they are lost.
The New AI: The Zero-Shot model is like a Universal Translator. It has read millions of different stories (data patterns) before. When you show it a new truck's data, it doesn't need to "study" it first. It instantly understands the pattern and can predict the future, no matter how the truck's behavior changes. It doesn't need to be retrained every time things shift.

3. How CLOUDADV Works: The "Decision Sandwich"

The system doesn't just spit out a number; it acts like a careful consultant.

Gathering Evidence: It looks at the computer's past usage (like checking the truck's logbook), predicts future needs using the Universal Translator, and checks the price list for different computer sizes.
The "Bounded" Context: It puts all this info into a structured "decision box." It tells the AI: "Here is the history, here is the forecast, here are the available options, and here are the safety rules (e.g., 'don't go below 85% capacity just to save a few dollars').".
The Recommendation: A Large Language Model (LLM) reads this box and suggests a move: "Downgrade to a smaller size," "Keep it as is," or "Upgrade."
The Safety Check: Before showing the engineer, the system double-checks its math. Did it suggest a computer that actually exists? Did it get the price right? If not, it fixes it.

4. The "Big Brain" vs. The "Fast Brain" Test

The researchers wanted to know if a smaller, cheaper AI could do the same job as a massive, expensive one.

The Big Brain (Offline): They used a super-powerful AI (Claude Opus) to generate the "gold standard" recommendations.
The Fast Brain (Online): They tested a smaller, faster AI (Qwen3.5) to see if it could match the Big Brain's advice in real-time.
The Result: The Fast Brain agreed with the Big Brain almost all the time (over 85% agreement). This means companies can use the cheaper, faster AI in production without losing quality.

5. The Results: Saving Money Without Crashing

They tested this on seven real-world computers.

The Savings: The system suggested changes that would drop the monthly bill from $1,503 to $708. That's a 53% savings (about $795 a month) just by picking the right size.
The Safety: They checked if the smaller computers would crash. The result? The "overshoot" (when the computer gets too full) happened very rarely. The worst case was only 1.5% of the time.
The Catch: The system is an advisor, not a robot driver. It gives the recommendation, but a human engineer must look at it and say, "Yes, that makes sense," before making the change. This is crucial because the AI can't know about a secret meeting next week where the company plans to launch a new product.

Summary

CLOUDADV is like a smart financial advisor for your cloud computers. It uses a "Universal Translator" AI to predict usage without needing to be retrained every time things change. It bundles the data, the forecast, and the safety rules into a clear recommendation, checks its own work, and suggests ways to cut costs by nearly half. The best part? It works so well that a smaller, cheaper AI can do the job just as well as the expensive one, making it practical for real-world use.

Technical Summary: CLOUDADV

Problem Statement

Cloud virtual machines (VMs) are frequently overprovisioned, leading to significant operational costs and resource inefficiency. Current provisioning practices often rely on conservative, static sizing based on anticipated peak demand, leaving resources underutilized for extended periods. While automated right-sizing tools exist, they typically rely on fixed lookback windows and percentile-based heuristics, lacking forward-looking forecasts or mechanisms for engineer feedback. Furthermore, traditional supervised forecasting models require per-VM retraining, revalidation, and redeployment cycles when workload behavior drifts, creating operational overhead and latency. The core challenge addressed is how to achieve decision-aligned instance sizing—where the forecast supports a safe and cost-effective provisioning action—under workload drift without the burden of maintaining separate, retrainable models for every tenant or horizon.

Methodology

The authors present CLOUDADV, an interactive, engineer-facing advisory system designed to generate VM sizing recommendations across day-, week-, and month-scale planning horizons. The system operates through a structured pipeline:

Telemetry and Preprocessing: The system ingests 1-minute interval CPU utilization and available memory data. Memory usage is derived by subtracting available memory from the current SKU's total capacity. Data is resampled to horizon-specific granularities (30-min, 6-hr, 12-hr) and imputed for missing values.
Zero-Shot Forecasting: Instead of training per-VM models, CLOUDADV employs zero-shot time-series foundation models (specifically Chronos-2 and TimesFM 2.5) to generate probabilistic forecasts. These models require no task-specific training, allowing them to adapt to workload drift without retraining triggers.
Bounded Decision Context Construction: Forecasts are not mapped directly to actions via closed-form rules. Instead, the system constructs a structured decision context ( $c_{i,t,H}$ $c_{i, t, H}$ ) containing:
- Current SKU metadata and pricing.
- Historical and forecasted utilization summaries (50th, 95th, and effective 95th percentiles).
- A filtered list of candidate SKUs restricted to a neighborhood of the current allocation (within a factor of $\delta=2.0$ ).
- Explicit sizing heuristics ( $\Gamma_H$ ) enforcing conservative headroom requirements ( $\tau_{cpu}=0.85, \tau_{mem}=0.80$ ).
Recommendation Generation: A Large Language Model (LLM) recommendation layer processes the bounded context to generate a structured recommendation (Action: maintain/downgrade/upgrade, Target SKU, Rationale).
- Reference Model: A higher-capacity model (Claude Opus 4.6) is used offline to generate reference recommendations.
- Production Model: A smaller model (Qwen3.5-35B) is evaluated online to assess alignment under latency and cost constraints.
- Verification: Outputs are verified against the underlying evidence to ensure identifiers and numeric values match the retrieved context before being presented to the engineer.

Key Contributions

Decision-Aligned Formulation: The paper reframes cloud instance sizing as a decision-aligned forecasting problem, prioritizing the preservation of the downstream provisioning decision over raw forecast error minimization.
CLOUDADV System Design: The authors design an interactive system that integrates zero-shot forecasting, structured evidence retrieval, and policy-constrained LLM recommendation generation.
Deployment-Aware Evaluation Pipeline: A novel framework is introduced to assess the alignment between a strong offline reference model and a smaller, deployable production model using identical bounded prompts.
Empirical Evaluation: The system is evaluated on seven production VMs using simulated cost savings, ex-post exceedance rates, and forecast accuracy metrics across multiple horizons.

Results

Cost Optimization: In a case study of seven production VMs, the reference recommendations reduced simulated monthly costs from approximately $1,503 to $708, yielding $795/month in savings (52.9%). Six of the seven VMs received downgrade recommendations, while one was retained.
Safety and Exceedance: The system maintained a low exceedance rate. The highest observed exceedance among downgraded cases was 1.5% (for VM R7). Three of the five downgrade cases with non-zero exceedance had rates below 0.2%. However, when exceedance occurred, it could be severe (e.g., CPU overflow up to 85.6 percentage points at the 95th percentile for R6).
Forecast Accuracy: Chronos-2 and TimesFM 2.5 achieved competitive performance across all horizons (Day, Week, Month) compared to classical (ARIMA, Prophet) and supervised (TSMixer) baselines, despite having no per-VM training.
Model Alignment: Agreement between the reference (Opus 4.6) and production (Qwen3.5-35B) models was high. Exact SKU agreement ranged from 78.6% to 100.0%, and action-level agreement ranged from 85.7% to 100.0%. Alignment was strongest at day and week horizons, with slight degradation at the month horizon, particularly near decision boundaries.
Engineer Audit: A stratified audit of 17 recommendations (approx. 10% of the set) resulted in a 100% pass rate, with 15 cases exactly matching an engineer-derived baseline and 2 cases deemed acceptable despite minor SKU variations.

Significance and Claims

The paper claims that zero-shot foundation models can effectively support decision-aligned provisioning in non-stationary cloud environments. The primary significance lies in demonstrating that:

Zero-shot models can induce recommendation patterns similar to supervised per-VM baselines without the operational burden of repeated retraining, revalidation, and redeployment.
A smaller, lower-cost production model can closely align with a stronger reference model when operating within a bounded, evidence-constrained decision context.
CLOUDADV offers a practical alternative for teams unable to maintain per-tenant supervised forecasting pipelines, providing forward-looking, forecast-grounded recommendations that complement retrospective utilization summaries.

The authors remain modest regarding generalizability, noting that the study is limited to seven VMs and that the system is intended as an advisory mechanism requiring engineer review, particularly for memory-sensitive workloads or systems undergoing significant changes. They acknowledge that the system is vulnerable to non-stationarity outside the historical context window (e.g., migrations, long business cycles).