⚡ electrical engineering

Safe and Near-Optimal Gate Control: A Case Study from the Danish West Coast

This paper presents a case study on Ringkoebing Fjord where a digital twin and Uppaal Stratego are used to learn an online gate controller that successfully satisfies safety requirements while maintaining near-optimal performance compared to human operators.

Original authors: Martin Kristjansen (Aalborg University), Kim Guldstrand Larsen (Aalborg University), Marius Mikučionis (Aalborg University), Christian Schilling (Aalborg University)

Published 2026-04-07

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Martin Kristjansen (Aalborg University), Kim Guldstrand Larsen (Aalborg University), Marius Mikučionis (Aalborg University), Christian Schilling (Aalborg University)

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine a giant, shallow bathtub (the Ringkøbing Fjord) sitting on the coast of Denmark. This bathtub is separated from the ocean by a wall with 14 giant sliding doors (the gates).

The people in charge of this bathtub have a very tricky job. They need to keep the water level in the tub just right:

Too high? The water floods the houses and gardens around the tub.
Too low? The fish can't swim between the ocean and the tub, and the birds lose their nesting spots.
Too much current? Boats can't safely enter the lock to get in or out.

Right now, human operators stand by these doors, looking at the weather and the water, and manually deciding when to slide the doors open or shut. But with climate change causing more storms and rising sea levels, this manual job is getting harder and riskier.

This paper is about teaching a computer to do this job better, faster, and safer. Here is how they did it, explained simply:

1. The "Digital Twin" (The Video Game Simulator)

Before letting a computer control the real, expensive gates, the researchers built a digital twin. Think of this as a hyper-realistic video game simulation of the fjord.

In this game, the computer can simulate what happens if it opens all 14 doors, or just one, or none at all.
It knows the rules: "If the wind is too weak, don't open the doors to let ocean water in," or "If the water levels are too different, don't open the doors or the pressure will break them."

2. The "Smart Student" (Reinforcement Learning)

The computer didn't just follow a rulebook; it learned like a student taking a test. This is called Reinforcement Learning.

The Student: The computer controller.
The Teacher: The digital twin.
The Lesson: The computer tries opening and closing doors thousands of times in the simulation.
- If it floods the houses, it gets a huge "bad grade" (penalty).
- If it lets fish swim freely, it gets a "good grade" (reward).
- If it opens and closes the doors too much, it gets a "wear and tear" penalty (because the doors are old and break easily).

The computer keeps playing this game over and over, getting smarter every time, until it figures out the perfect strategy to keep the water level safe while letting fish and boats pass.

3. The "Weather Forecast" Trick

One big problem is that the weather changes. A plan made for today might be bad for tomorrow.

The Solution: The researchers taught the computer to look at the weather forecast (like a 3-day outlook) while it was learning.
The "Online" Update: Since forecasts change, the computer doesn't just learn once and forget. Every 6 hours, when a new forecast arrives, the computer quickly re-learns its strategy for the next few days. It's like a driver checking the GPS every time a new traffic jam is reported, rather than sticking to an old map.

4. The Results: The Computer Wins

The researchers tested their new "Smart Computer Controller" against the old "Human Rulebook" (the baseline) using three different weather scenarios: normal days, stormy days, and calm days.

Safety: The human rulebook failed. In the simulations, it let the water get too high or too low, risking floods or drying out the fish. The computer controller never failed. It kept the water in the "safe zone" 100% of the time.
Fish & Boats: The computer was just as good as the humans at letting fish swim and boats pass.
The Trade-off: The only downside? The computer opened and closed the doors a few more times than the humans did. This is because the computer was so desperate to keep the water level perfect that it made tiny adjustments constantly.

The Big Picture

This paper shows that we can use AI and simulations to manage critical infrastructure like dams and floodgates. Instead of relying on tired humans guessing the best move, we can have a "digital twin" that learns from the future (forecasts) and acts instantly to keep our towns safe and our wildlife happy.

In short: They taught a computer to play a high-stakes game of "keep the water level just right," and it learned to play better than the humans who have been doing it for years.

1. Problem Statement

The paper addresses the operational challenges of the Ringkøbing Fjord, a 290 km² inland water basin on the Danish west coast connected to the North Sea by 14 controllable gates and a sluice lock. Currently, gate operations are managed manually by human operators based on general principles. This manual approach faces several critical challenges:

Conflicting Objectives: Operators must balance safety (preventing flooding or drying out), environmental needs (fish migration), and infrastructure usage (maritime traffic).
Climate Change: Rising sea levels and increased precipitation reduce the time windows available for draining the fjord, making manual control increasingly difficult.
Safety Constraints: The water level must remain within a strict range (0.00 m to 0.25 m relative to DVR90) to protect wildlife and property. Additionally, gates cannot operate if the water level difference exceeds 1 m (structural risk) or if wind speeds are too low during inflow (mixing requirements).
Performance Trade-offs: Opening gates for fish migration conflicts with keeping them closed for boat safety; minimizing gate wear conflicts with maintaining optimal water levels.

The goal is to develop an automated, online learning-based controller that ensures safety while optimizing performance metrics (fish migration, boat wait times, and gate wear) under uncertain weather conditions.

2. Methodology

The authors propose a framework combining Digital Twins and Online Reinforcement Learning (RL) using the tool UPPAAL STRATEGO.

A. System Modeling (Digital Twin)

The physical system is modeled as a network of five stochastic hybrid automata:

Fjord Water Level: Modeled using a first-order Ordinary Differential Equation (ODE) derived from fluid dynamics principles. The change in water level depends on:
- Gate Flow ( $Q_{gates}$ ): Calculated based on the cross-sectional area of open gates, the height difference between sea and fjord, and flow constants ( $K_{inflow}$ and $K_{outflow}$ ).
- Stream Flow ( $Q_{streams}$ ): Modeled using historical monthly averages of freshwater inflow.
- Sea Level ( $h_s$ ) and Wind Speed: Derived from weather forecasts.
Controller: A hybrid automaton that decides the number of open gates (0, 1, or 14) every 10 minutes based on sensor readings and forecasts.
Boat Traffic: A stochastic model (exponential distribution) simulating boat arrivals.
Safety & Performance Monitors: Automata tracking time spent outside safe ranges, boat waiting times, and periods where fish migration is possible but blocked.

B. Online Reinforcement Learning

Approach: The system uses Q-learning with partition refinement. This allows the application of RL to continuous state spaces (water levels) by discretizing them during the learning process.
Online Strategy: Instead of training a single static policy, the system operates in an online loop:
1. A new weather forecast (valid for 3 days) is received.
2. The digital twin is updated with the new forecast.
3. A new strategy is learned for the upcoming 3-day horizon (approx. 60 minutes of computation).
4. The controller executes the strategy for a period (e.g., 6 hours) until a new forecast arrives, at which point the process repeats.
Objective Function: The learning query minimizes a weighted cost function:
$Cost = w_1(\text{Time out of range}) + w_2(\text{Boat wait time}^2) + w_3(\text{Gate operations}) + w_4(\text{Time fish blocked})$
- $w_1$ (Safety) is heavily weighted ( $10^6$ ) to ensure safety constraints are prioritized over performance.

3. Key Contributions

Digital Twin Implementation: Creation of a high-fidelity simulation model of the Ringkøbing Fjord within UPPAAL STRATEGO, integrating fluid dynamics, weather forecasts, and stochastic traffic.
Online Learning Framework: Demonstration of an online RL approach that adapts to changing environmental conditions (sea level rise, weather updates) by re-learning strategies periodically, overcoming the limitations of static controllers.
Safety-Guaranteed Control: The learned controllers strictly adhere to safety constraints (water level range, wind speed limits, and structural difference limits), whereas the baseline human-like controller frequently violates them.
Weight Sensitivity Analysis: A systematic investigation into the trade-offs of the safety weight ( $w_1$ ), showing that while high weights ensure safety, they can negatively impact other performance metrics (like boat wait times), highlighting the need for careful tuning.

4. Experimental Results

The authors evaluated the learned controllers against a deterministic Baseline Controller (mimicking human guidelines) across three 3-day scenarios: Normal Waters, Low Waters, and High Waters.

Safety Performance:
- Learned Controllers: Achieved 100% safety compliance across all scenarios. The water level never left the [0.00, 0.25] m range.
- Baseline Controller: Violated safety requirements in all scenarios. In the "Normal Waters" scenario, it spent 36.8% of the time outside the safe range. In "High Waters," it breached the upper limit by up to 2 cm.
Performance Metrics:
- Fish Migration: Learned controllers performed similarly or slightly better (99.05% vs 95.70% in normal waters).
- Boat Waiting Time: Both controllers performed comparably (approx. 8.8–9.0 minutes max wait).
- Gate Operations: The learned controllers performed more operations (e.g., 40.3 vs 26.1 in normal waters) to maintain safety, indicating a trade-off where the algorithm prioritizes safety over minimizing wear.
Weight Tuning: Reducing the safety weight ( $w_1$ ) from $10^6$ to $10^3$ reduced gate operations but introduced minor safety violations (approx. 0.55‰ of time outside range). This suggests a tunable balance between strict safety and operational efficiency.

5. Significance and Future Work

Significance: This work demonstrates that UPPAAL STRATEGO can effectively learn near-optimal control strategies for complex, continuous physical systems in under an hour. It proves that online learning can outperform human-guided heuristics in critical infrastructure management, particularly in adapting to climate change-induced variability.
Future Directions:
- Granular Control: Moving beyond the 3-action limit (0, 1, 14 gates) to fine-grained control to reduce wear.
- Salinity Management: Explicitly modeling salinity levels to ensure ecological health, rather than relying on wind speed as a proxy.
- Water Surface Dynamics: Refining the model to account for wind-induced water level differences across the fjord (up to 70 cm).
- Shielding: Integrating formal "shields" to mathematically guarantee that no unsafe actions are ever taken by the learned policy.
- Pareto Optimization: Computing Pareto fronts for the cost function weights to help stakeholders make informed political/economic decisions on trade-offs.

In conclusion, the paper presents a robust, data-driven approach to managing critical water infrastructure, successfully balancing strict safety requirements with operational efficiency through the use of digital twins and online reinforcement learning.