XGBoost-Based Optimal Transport Dataset Distance-Guided Temporal Domain Transfer for Flood Prediction Under Concept Drift

Floods are among the most destructive forces on Earth, reshaping landscapes and upending lives with terrifying speed. For decades, scientists have tried to predict these events using mathematical models that rely on historical weather patterns and river levels. These models work well when the climate is stable, but the world is changing. Rainfall patterns are shifting, soil conditions are altering, and urban development is changing how water moves across the land. In the language of data science, this shifting reality is called "concept drift," a phenomenon where the rules that governed the past no longer apply to the present. When a prediction model is trained on old data, it can become blind to these new realities, leading to dangerous failures in early warning systems. The challenge for modern researchers is not just to build a smart model, but to build one that can learn, adapt, and evolve as the environment changes around it.

A team of researchers at Victoria University in Kampala has tackled this problem by creating a new kind of flood prediction system that learns from its own mistakes and adjusts to changing conditions in real time. They combined three powerful tools into a single framework: a highly accurate prediction engine, a sensitive alarm system for detecting change, and a mathematical ruler for measuring how different the new world is from the old one. Their goal was to move beyond static models that sit still and wait to be retrained, creating instead a system that can continuously update itself to stay accurate. By testing their approach on a massive dataset of historical flood records, they demonstrated that their adaptive model could maintain high accuracy even when the weather patterns shifted dramatically, outperforming traditional methods that failed to notice the change.

The researchers began with a vast collection of historical data containing information about rainfall, river flow, humidity, temperature, and soil moisture, all linked to whether a flood actually occurred. They split this data into a timeline, treating it not as a random pile of numbers, but as a story unfolding over time. The first part of their system acts as a watchdog, constantly scanning the incoming stream of weather data. This component, known as adaptive windowing, compares the current weather conditions against a rolling window of recent history. It is designed to ignore minor, everyday fluctuations but to sound an alarm the moment a statistically significant shift occurs. In their tests, this system successfully identified a major change in environmental conditions, signaling that the old rules were no longer sufficient to describe the new reality.

Once the alarm was raised, the system did not simply discard everything it had learned and start over. Instead, it engaged a second component to measure exactly how different the new conditions were from the old ones. This step uses a method called optimal transport dataset distance, which essentially calculates the "distance" between the old data and the new data. Imagine trying to move a pile of sand from one spot to another; this method calculates the least amount of effort required to reshape the old pile to match the new one. If the distance is small, the system knows the new conditions are similar enough to the old ones that it can simply tweak its existing knowledge. If the distance is large, it knows a more significant update is needed. This measurement is crucial because it prevents the model from blindly applying old lessons to a completely different environment, a mistake that often leads to poor predictions.

With this understanding of how much the world had changed, the system then adjusted its prediction engine, which is built on a powerful algorithm called XGBoost. This engine is known for its ability to find complex, non-linear relationships between weather variables and flood outcomes. When the system detected a small shift, it adapted the existing model by incorporating the new data, preserving the valuable knowledge it had already gained. When the shift was large, it initiated a more thorough update. In their experiments, the researchers observed that when a major change occurred in the data, the prediction error of a standard, unchanging model spiked dramatically, becoming almost useless. However, their adaptive system detected this shift, measured the difference, and adjusted itself. Within moments, the error rate dropped back down to a very low level, proving that the model had successfully relearned the new rules of the environment.

The results of this study were striking. The adaptive model achieved an accuracy of 88.7% in predicting flood events, significantly outperforming both standard statistical models and a version of the prediction engine that was not allowed to adapt. It correctly identified flood events 91.6% of the time, a critical metric for disaster preparedness where missing a flood is far worse than a false alarm. Perhaps most importantly, the system achieved this adaptability with very little computational cost. The time required to detect a change and update the model remained under one second for most of the test periods. This speed suggests that such a system could be deployed in real-world early warning networks, where decisions must be made instantly as new weather data arrives.

The researchers also noted that the dataset they used was heavily skewed, with non-flood days vastly outnumbering flood days. This is a common reality in flood prediction, where true disasters are rare events. Despite this imbalance, the adaptive system managed to learn the subtle signs of an approaching flood without being overwhelmed by the quiet days. The study highlights that the key to reliable prediction in a changing climate is not just having a smart algorithm, but having a system that knows when to listen to the past and when to listen to the present. By combining drift detection with a precise measurement of environmental change, the team created a framework that does not just predict the future, but evolves to understand it.

While the results are promising, the authors are careful to note that their work was conducted using historical data. The true test will come when this system is deployed to monitor live, real-time sensor streams across different geographical regions. The framework provides a solid foundation for continuous flood monitoring, but future work will need to validate its performance in the chaotic, unpredictable flow of actual river basins. For now, the study offers a clear path forward: a way to build flood prediction systems that are not static monuments to the past, but living tools capable of surviving the changing climate of the future.