An Auto-Scaling Approach for Serverless Environments Based on a Multi-Expert Consensus Mechanism
This paper proposes a dependency-aware serverless autoscaling framework that integrates graph-based bottleneck identification, a multi-model consensus forecasting mechanism, and cost-aware control to achieve high prediction accuracy, reduced errors, and lower infrastructure costs while maintaining performance targets.
Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
Imagine the internet as a massive, bustling city where millions of tiny, invisible workers—called "functions"—rush to complete tasks the moment you click a button. In the old days, these workers lived in big, permanent offices (servers) that had to be staffed 24/7, whether they were busy or asleep. But a newer, smarter way of building the internet, called "serverless computing," lets these workers pop into existence only when needed and vanish when they're done, saving a fortune on rent. However, this system has a tricky problem: sometimes the city gets hit by a sudden, chaotic rush of requests, like a flash mob. If the system waits to see the crowd before hiring more workers, the first few people get stuck waiting in line (a "cold start"). If it hires too many too fast, it wastes money. The real puzzle is that these workers often depend on each other; if one gets stuck, the whole chain of work grinds to a halt. Researchers are trying to figure out how to predict these rushes and hire the right number of workers at the right time, without wasting money or making people wait.
This paper introduces a clever new "traffic controller" for these serverless cities. Instead of watching every single worker individually, the authors suggest looking at the city's map to find the most important intersections. They treat the application like a web of connections, using a math trick called "degree centrality" to spot the few functions that are connected to the most other parts of the system. These are the "bottlenecks"—the busy intersections where traffic jams are most likely to start. Once they identify these critical spots, the system doesn't just guess what will happen next; it asks three different "experts" (computer models named MLP, LSTM, and CNN) to predict the traffic. Each expert sees the data differently: one looks at patterns, another at time sequences, and the third at local details. Instead of trusting just one, the system uses a "consensus" method, similar to a jury, where the experts vote based on how well they've performed in the past. This creates a stable, reliable prediction. Finally, before making a move, the system checks the price tag, ensuring that hiring more workers won't cost too much and that firing workers won't cause a cold, expensive delay.
The researchers tested this idea using real-world data traces and found that their "multi-expert" approach works significantly better than older methods. They discovered that relying on a single prediction model is risky, much like trusting only one weather forecaster. By combining the three different models, they achieved a prediction accuracy of about 99.06% with their best model (LSTM), which is a huge improvement over older, unsupervised methods that only managed about 49.5% accuracy. The study explicitly rules out the idea that simple, unsupervised clustering (grouping data without a teacher) is enough to make good scaling decisions; the data showed that these methods failed to capture the direction needed to know when to scale up or down. Furthermore, the system proved that by focusing only on the critical "bottleneck" functions rather than every single function, it could make smarter decisions.
In terms of money, the results were quite tangible. When the researchers simulated their system on different cloud platforms, it saved real cash. For example, on AWS Lambda, the cost dropped from $0.70 to $0.47, and on Google Cloud Run, it fell from $9.48 to $6.36. Across the board, the system saved a total of $5.55 compared to a standard, reactive system that just waits for problems to happen. The authors suggest that this approach is a practical way to balance speed and cost, but they are careful to note that these results come from simulations and experiments on specific datasets, not a permanent fix for every possible scenario. They also point out that while their method is robust, it still relies on the quality of the data it's fed. The paper concludes that while this "dependency-aware" system is a strong step forward, there is still room to grow, perhaps by using even more advanced math like graph neural networks or by testing it on live, real-world platforms in the future.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.