Model Predictive Control and Moving Horizon Estimation using Statistically Weighted Data-Based Ensemble Models

This paper proposes a novel model predictive control framework that utilizes a Mahalanobis distance-based weighted ensemble of data-driven models and a moving horizon estimation observer to effectively control complex systems across multiple operating conditions.

The Gist

Imagine you are the captain of a massive, complex ship (a district heating system) trying to navigate through changing weather. Sometimes the water is calm and warm (summer conditions); other times, it's stormy and freezing (winter conditions). To steer this ship efficiently and safely, you need a navigation team that can predict exactly where the ship will be in the next few hours.

This paper introduces a new way to build that navigation team and a new way to steer the ship. Here is the breakdown in simple terms:

The Problem: One Map Isn't Enough

Usually, engineers try to build one single "black box" model (like a super-smart AI) to predict how the ship behaves in all conditions. But just like a single map can't perfectly show both a desert and an iceberg, one model often gets confused when the weather changes. It might predict the ship will move fast in a storm when it actually slows down, leading to poor decisions or safety violations.

The Solution: A Team of Specialists (Ensemble Models)

Instead of one generalist, the authors suggest hiring a team of specialists.

• Specialist A is an expert on "Summer Conditions." They trained only on summer data.
• Specialist B is an expert on "Winter Conditions." They trained only on winter data.

When you need a prediction, you don't just pick one; you ask both for their opinion and combine their answers. But the tricky part is: How much do you trust each specialist?

The Innovation 1: The "Statistical Compass" (Mahalanobis Distance)

In the past, people would either:

1. Take the average of both opinions (50/50), which is often wrong.
2. Ask, "Who was right in the past?" and trust them more. But in a control system, you are looking into the future, and you don't know the future yet.

The authors propose a new rule based on Mahalanobis Distance. Think of this as a statistical compass.

• The system looks at the current weather (the inputs, like temperature and load).
• It asks: "How statistically similar is today's weather to the data Specialist A learned from? How similar is it to Specialist B?"
• If today looks very much like a "Summer Day," the compass gives Specialist A a huge vote (high weight) and Specialist B a tiny vote.
• Crucially, this compass works only on the inputs (what you plan to do next), not on future outputs (which you don't know yet). This allows the system to smoothly shift trust between specialists as the weather changes during the prediction window.

The Innovation 2: The "Memory Lane" Observer (Moving Horizon Estimation)

There is a second problem. These AI specialists (specifically Gated Recurrent Units, or GRUs) have an internal "memory" or "state" that helps them make predictions. However, this memory is invisible to the captain; you can only see the outside temperature and water flow.

If the captain guesses the memory wrong, the prediction will drift off course.

• Old Way: Just let the model run on its own (Open Loop). If it makes a small mistake, the error grows bigger and bigger.
• New Way (MHE): The authors built a "Memory Lane" observer. Instead of just looking at the last second, it looks back at the last 50 steps of history. It asks: "Given everything that happened in the last 50 minutes, what must the internal memory have been to produce these results?"
• It then adjusts the memory to fit the history perfectly before making the next prediction. This is like a detective reconstructing a crime scene to understand the current situation better.

The Result: A Smoother, Cheaper Ride

The authors tested this on a real-world heating system (the AROMA system) that switches between summer and winter modes. They compared their new method against:

• Rule-based: A simple, rigid set of rules (like a human following a manual).
• Average: Trusting both specialists equally.
• Least Squares: Trusting whoever was right most recently.
• Fixed Mahalanobis: Using the compass, but only looking at the current moment, not the future.
• Their Method (MD-2): Using the compass to adjust trust throughout the entire future prediction window.

The Findings:

1. Savings: Their method saved the most money (economic performance) because it could anticipate changes in the weather better than the others.
2. Safety: It made the fewest mistakes regarding safety limits (like water getting too hot or too cold).
3. Accuracy: The "Memory Lane" observer significantly reduced the errors in the model's internal predictions, making the whole system more reliable.

In a Nutshell

This paper teaches us how to control complex systems by using a team of specialized AI models rather than one generalist. It uses a statistical compass to decide who to trust based on current conditions, and a historical detective to fix the AI's internal memory. The result is a system that is cheaper to run and safer to operate when conditions change.

Technical Summary

Technical Summary: Model Predictive Control and Moving Horizon Estimation using Statistically Weighted Data-Based Ensemble Models

Problem Statement
Model Predictive Control (MPC) is a dominant strategy for managing complex systems, yet its performance relies heavily on the accuracy of the underlying prediction model. While data-based models (e.g., neural networks) offer advantages in development and data availability, a single black-box model often fails to maintain prediction accuracy across diverse operating conditions. Furthermore, standard ensemble learning approaches, which combine multiple specialized models, typically rely on static weighting or weighting based on past output errors. These methods are suboptimal for MPC because they cannot dynamically adjust weights across the prediction horizon where future state and output measurements are unavailable. Additionally, estimating the internal states of data-based ensemble models (which often lack physical meaning) while enforcing state constraints remains an unresolved challenge in the literature.

Methodology
The paper proposes a unified framework integrating Model Predictive Control (MPC) and Moving Horizon Estimation (MHE) for systems operating under multiple conditions. The core of the approach involves an ensemble of data-based models (specifically Gated Recurrent Units, or GRUs), where each expert is trained on a dataset corresponding to a specific operational domain.

1. Mahalanobis Distance-Based Weighting:
   Instead of static averaging or output-dependent weighting, the authors propose a novel combination rule where ensemble weights are functions of the system input. The weight assigned to each expert model $M^{[i]}$ at a specific time step is inversely proportional to the statistical Mahalanobis distance between the current (or future) input vector and the training input data of that specific expert.
   $$\lambda^{[i]}_{MD}(u(h)) = \frac{1 / T^2(u(h), u^{[i]})}{\sum_{i=1}^{n} 1 / T^2(u(h), u^{[i]})}$$
   This allows the weights to vary dynamically across the prediction window based solely on the known sequence of future inputs, enabling the MPC to anticipate shifts in operating conditions.

2. MHE-Based State Observer:
   To address the challenge of state estimation for data-based models, a Moving Horizon Estimation (MHE) observer is developed for each expert in the ensemble. The MHE problem minimizes the error between the model's predicted outputs and the actual measured plant outputs over a finite past horizon. Crucially, this formulation explicitly incorporates state constraints (e.g., amplitude bounds on hidden states), ensuring that the estimated states used to initialize the MPC are physically consistent with the model's internal limitations.

Key Contributions
The paper makes two primary contributions:

• A Novel MPC Formulation: It introduces an MPC regulator that utilizes a data-based ensemble where combination weights are determined by the statistical proximity (Mahalanobis distance) between the optimized inputs and the training data of each expert. This allows for input-dependent, horizon-varying weights without requiring future output measurements.
• A Constrained State Observer: It proposes a new observer design for ensemble models based on MHE. This method enables the accurate estimation of states for all ensemble experts while strictly enforcing state constraints, a capability previously unaddressed for ensemble models in the literature.

Numerical Results
The framework was validated on a benchmark District Heating System (DHS) featuring multiple operating conditions (simulating summer and winter load profiles). The system was controlled using an MPC with a 72-step horizon, comparing four weighting strategies:

• AV: Simple arithmetic averaging.
• LS: Least-squares optimization based on past 100 time steps.
• MD-1: Mahalanobis-based weights fixed to the value at the previous time step.
• MD-2: Mahalanobis-based weights optimized as a function of inputs across the entire prediction horizon.

Results demonstrated that the proposed MD-2 strategy achieved the best economic performance (lowest cost) and the fewest constraint violations compared to the other MPC strategies and a rule-based baseline. Specifically, while AV and LS strategies exhibited constraint violations, the Mahalanobis-based approaches maintained feasibility. The MD-2 approach outperformed MD-1 by reacting earlier to future condition changes, highlighting the benefit of horizon-varying weights. Additionally, the MHE-based observer significantly reduced the one-step output prediction error for the individual models compared to open-loop state propagation.

Significance and Claims
The authors claim that their methodology significantly outperforms standard averaging and optimization-based weight selection methods in terms of both economic performance and constraint satisfaction. They assert that the proposed Mahalanobis-based combination rule is a robust solution for handling varying operating conditions within an MPC framework, as it relies only on available input data rather than unavailable future outputs. Furthermore, the paper highlights that the MHE-based observer effectively addresses the critical issue of state estimation for data-based models under constraints, leading to improved closed-loop performance. The computational burden of the proposed method is noted to remain reasonable despite the added complexity. Future work is identified as integrating this framework into active learning settings for informative data acquisition.