🤖 machine learning

Harmonica: A Self-Adaptation Exemplar for Sustainable MLOps

This paper introduces Harmonica, a self-adaptation exemplar built on the HarmonE approach that utilizes a MAPE-K loop to monitor sustainability metrics and automatically trigger architectural tactics, thereby enabling sustainable and stable operation of Machine Learning-enabled systems within MLOps pipelines.

Original authors: Ananya Halgatti, Shaunak Biswas, Hiya Bhatt, Srinivasan Rakhunathan, Karthik Vaidhyanathan

Published 2026-02-03

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Ananya Halgatti, Shaunak Biswas, Hiya Bhatt, Srinivasan Rakhunathan, Karthik Vaidhyanathan

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a very smart, self-driving delivery robot. You want it to be fast, accurate, and not run out of battery too quickly. But the world is messy: sometimes the roads are crowded, sometimes it's raining, and sometimes the robot gets tired or confused. If you just set the robot and walk away, it might crash, waste energy, or stop working entirely.

This paper introduces Harmonica, a "smart manager" designed to keep these AI systems running smoothly, efficiently, and sustainably over the long term. Think of it as a personal trainer and traffic controller rolled into one, specifically for AI systems that use MLOps (the automated assembly lines that build and run AI).

Here is how it works, broken down into simple concepts:

1. The Problem: The "Set It and Forget It" Trap

Usually, when companies build AI, they use a process called MLOps. It's like a factory assembly line that builds the AI, tests it, and puts it to work. The problem is that once the AI is working, the factory stops paying attention.

The Issue: If the environment changes (like a sudden storm or a new type of traffic), the AI might start making mistakes, using too much electricity, or costing too much money to run.
The Gap: Existing tools are great at building the AI, but they don't have a built-in mechanism to say, "Hey, things are getting weird, let's adjust!" without a human having to step in and fix it manually.

2. The Solution: Harmonica (The Self-Adapting Manager)

Harmonica is a tool that sits on top of your AI system and watches it like a hawk. It is built on a concept called MAPE-K, which is a fancy way of describing a four-step loop that never sleeps:

Monitor (The Eyes): It constantly watches the AI. Is it accurate? Is it using too much battery? Is it confused by the data?
Analyze (The Brain): It compares what it sees against a set of rules you set beforehand. For example: "If the AI gets less than 80% accurate, or if it uses more than 50 watts of power, something is wrong."
Plan (The Strategist): If the rules are broken, it decides what to do. Should it switch to a simpler, faster model? Should it take a break to retrain itself?
Execute (The Hands): It automatically makes the change. It swaps the model, adjusts the settings, or triggers a retraining session without you touching a button.

The "K" in MAPE-K stands for Knowledge. This is the tool's memory bank. It remembers your goals, the history of what the AI has done, and the different versions of the AI models it can switch between.

3. The "Menu" of Options

Harmonica doesn't just guess; it has a menu of different AI models to choose from, ranging from:

The Sprinter: A small, fast, low-energy model (great for simple tasks, but maybe less accurate).
The Marathon Runner: A big, powerful, high-energy model (very accurate, but eats a lot of battery).

Harmonica's job is to pick the right runner for the current race conditions. If the road is clear, it picks the Sprinter to save energy. If the road is foggy and tricky, it switches to the Marathon Runner to ensure safety.

4. How They Tested It (The Race Tracks)

The researchers tested Harmonica in two different "race tracks" to see if it could handle different types of AI jobs:

Track 1: Traffic Prediction (Time Series): They used it to predict traffic flow. They simulated a situation where traffic patterns suddenly changed (like a new road being built).
- Result: Harmonica noticed the change, realized the old model was struggling, and automatically switched to a better model. It kept the predictions accurate while saving energy compared to just using the biggest, most powerful model all the time.
Track 2: Object Detection (Computer Vision): They used it to spot cars and pedestrians in video footage (like a self-driving car's eyes).
- Result: It successfully ran different versions of the vision software, showing it could handle the heavy lifting of seeing the world and adapting when the lighting or scenery changed.

5. What Real People Thought (The User Study)

The researchers asked 12 people (students, researchers, and industry pros) to try out Harmonica.

The Verdict: They loved it.
Why? They found it very easy to install. They liked the Live Dashboard (a screen showing real-time graphs of energy and accuracy) because it let them "see" the AI thinking and adapting.
The Feeling: It felt like having a tool that made complex AI management feel simple and transparent.

The Bottom Line

Harmonica is a reusable "exemplar" (a working prototype) that proves you can build AI systems that take care of themselves. It separates the "what" (your goals, like "save energy") from the "how" (the technical code that switches models).

It doesn't invent new AI models; instead, it provides the orchestra conductor that tells the existing AI models when to play loud, when to play soft, and when to switch instruments, ensuring the music (the system) keeps playing sustainably, no matter how the environment changes.

Where to find it: The authors have made the code and a video walkthrough available online so anyone can try it out and see how it works.

Technical Summary: Harmonica – A Self-Adaptation Exemplar for Sustainable MLOps

Problem Statement
Machine Learning Enabled Systems (MLS) frequently operate in dynamic environments where runtime uncertainties—such as distribution shifts, energy fluctuations, and performance degradation—can compromise system viability. While Machine Learning Operations (MLOps) provides structured processes for development and deployment, current practices primarily emphasize reliability and automation. They lack systematic mechanisms to address broader sustainability dimensions (environmental, economic, and social) or to provide runtime self-adaptation that balances trade-offs between predictive accuracy, energy consumption, and operational costs. Existing self-adaptive approaches are often tightly coupled case study prototypes or focus solely on inference-time model switching, leaving a gap for reusable, end-to-end exemplars that operationalize sustainability-aware adaptation within MLOps pipelines.

Methodology and Architecture
The paper presents Harmonica, a self-adaptation exemplar built upon the HarmonE architectural approach. Harmonica is designed as a managing system that wraps existing MLOps pipelines without requiring modifications to the underlying training or inference code. It employs a MAPE-K loop (Monitor, Analyzer, Planner, Executor, Knowledge) to separate high-level adaptation policies from low-level tactic execution.

The system architecture comprises three primary components:

Frontend: A lightweight web application (HTML, CSS, JavaScript) serving as the interaction layer. It includes:
- Approach Manager: Allows selection of adaptation logic (e.g., model switching, HarmonE approach, or custom strategies).
- Threshold Configurer: Enables users to define design-time adaptation boundaries (e.g., upper/lower limits for energy, accuracy, or cost).
- Live Dashboard: Provides real-time visualization of sustainability metrics and adaptation events.
Managing System: A Python Flask server that orchestrates the MAPE-K loop. It maintains a Knowledge base containing:
- Repositories: Data, System Logs, Sustainability Goals (policy files), Current Models, and Versioned Models.
- MAPE Components:
  - Monitor: Collects runtime metrics (accuracy, confidence, energy) from system logs.
  - Analyzer: Evaluates metrics against user-defined boundaries to detect drift or violations.
  - Planner: Selects adaptation tactics based on stored policies when violations occur.
  - Executor: Executes the chosen tactic (e.g., switching models, triggering retraining) on the Managed System.
Managed System: The target ML pipeline (implemented in Python) containing the Inference Subsystem (data ingestion, preprocessing, inference engine) and the Training Subsystem (triggered for retraining/fine-tuning).

Key Contributions
The paper claims four main contributions:

Configurable Managing System: A framework that executes adaptation logic over any compatible MLOps pipeline, supporting both inference and retraining workflows.
User-Defined Configuration: The ability for users to upload custom datasets and define sustainability goals and adaptation boundaries via an intuitive interface.
Built-in Visualization: Interactive dashboards that display live sustainability and performance metrics, facilitating the analysis of adaptation effects.
Offline Analysis Support: Provision of downloadable telemetry and log data for repeatable empirical studies.

Empirical Evaluation and Results
Harmonica was evaluated through two use cases:

Time Series Regression (Traffic Prediction): Using California PeMS data, the system compared static models (Linear Regression, SVM, LSTM), a naive switching heuristic, and the HarmonE approach. Results indicated that while the static LSTM offered the highest accuracy ( $R^2$ 0.9171), it incurred significantly higher energy costs. The HarmonE approach achieved a balanced $R^2$ of 0.8938 with moderate energy usage (42,645 J), demonstrating the ability to adapt to runtime uncertainties and avoid unnecessary interventions compared to static or naive switching baselines.
Computer Vision (Object Detection): Using the BDD100K dataset and YOLOv8 variants (n, s, m), the system demonstrated its capability to execute and evaluate different model variants. While the full HarmonE logic was not applied due to domain-specific generalization needs, the exemplar successfully facilitated comparative studies of adaptation strategies in a computer vision setting.

A User Study involving 12 participants (students, researchers, and industry practitioners) assessed usability. Participants rated the system highly for ease of installation (mean 4.58/5) and the clarity of the monitoring dashboard (mean 4.41/5). The likelihood to recommend the tool was 4.66/5. Qualitative feedback highlighted the value of real-time telemetry in understanding adaptation logic, though participants suggested improvements for on-screen tooltips explaining specific tactics.

Significance and Claims
The paper positions Harmonica not as a novel adaptation algorithm, but as a practical and reusable foundation for the research community. Its significance lies in:

Operationalizing Sustainability: It bridges the gap between theoretical sustainability goals and practical MLOps implementation by providing a tool to study trade-offs across the full ML lifecycle.
Lowering Barriers to Entry: By offering a pre-built, configurable exemplar, it eliminates the need for researchers to reimplement monitoring pipelines and adaptation logic, thereby enhancing reproducibility.
Enabling Controlled Experimentation: It allows for the systematic exploration of alternative adaptation strategies and the inspection of system behavior under varying operating conditions without modifying core ML code.

The authors maintain a modest stance, acknowledging that while the current implementation uses pyRAPL for energy measurement (limited to Intel hardware), the framework is designed to be extended with platform-agnostic energy proxies in future work to better align with diverse real-world deployment contexts.