Artificial Intelligence for Climate Adaptation: Reinforcement Learning for Climate Change-Resilient Transport

This paper proposes a novel reinforcement learning-based decision-support framework that outperforms traditional optimization methods by discovering coordinated, long-term adaptation pathways for urban transport systems to effectively balance investment costs against climate-induced flood risks under deep uncertainty, as demonstrated in a case study of Copenhagen.

The Gist

The Big Picture: Planning for a Wetter Future

Imagine your city is a giant, complex game of Jenga. For decades, the tower has been stable. But now, the climate is changing, and it's starting to rain much harder and more often. If you keep stacking blocks (building roads and houses) without accounting for the extra rain, the tower is going to wobble and eventually collapse.

This paper is about how to play that game smarter. The authors, a team of researchers from Denmark and Ireland, are asking: "How do we build a city that can survive the heavy rains of the next 75 years without going broke?"

They focus on Copenhagen, a city that has already suffered massive damage from "cloudbursts" (sudden, violent storms). Instead of just guessing what to do, they built a super-smart AI coach using a technique called Reinforcement Learning (RL).

---

The Problem: Why Old Plans Don't Work

Traditionally, city planners try to solve this like a math equation. They look at the data, pick the best solution, and stick to it. But climate change is messy. It's like trying to plan a road trip 50 years in the future when you don't know if the weather will be sunny, stormy, or a mix of both.

• The "Static" Trap: If you build a giant sea wall today based on today's weather, it might be useless in 2050 when the rain patterns change.
• The Complexity: You have to balance the cost of building things (like better drains) against the cost of damage (like flooded cars and cancelled trips). Doing this for 29 different neighborhoods over 76 years is too hard for a human brain or a standard computer to calculate perfectly.

The Solution: The AI Coach (Reinforcement Learning)

The researchers treated the city like a video game.

1. The Player (The AI): This is the Reinforcement Learning agent. Its goal is to win the game by keeping the city dry and the economy moving.
2. The Environment: The game board is Copenhagen. The "enemies" are rainstorms.
3. The Moves (Actions): The AI can choose to install different flood defenses in different neighborhoods. Think of these as power-ups:
   • Soakaways: Holes in the ground to drink up rain.
   • Bioretention Planters: Gardens that soak up water.
   • Storage Tanks: Big underground buckets to hold water.
   • Porous Asphalt: Roads that let water seep through instead of pooling.
4. The Score (Reward): The AI gets points (or loses points) based on two things:
   • Cost: How much money did we spend on building these defenses?
   • Damage: How much money did we lose because roads were flooded, people were stuck in traffic, or trips were cancelled?

The AI plays the game millions of times. Every time it makes a bad move (spending too much money on a tank that never gets used), it learns. Every time it finds a perfect balance (spending a little now to save a lot later), it gets a high score. Eventually, it learns the perfect strategy.

What Did the AI Discover?

When the AI played the game for Copenhagen (from 2024 to 2100), it found some surprising and smart patterns that traditional planners might have missed:

• It's Not "All or Nothing": The AI didn't just build defenses everywhere at once. It learned to time the investments. It waited to see how the weather was behaving before spending money, then acted quickly when needed.
• The "Goldilocks" Strategy: The AI found that the best plan isn't the most expensive one, nor the cheapest one. It's the one that spends just enough to prevent the worst damage.
• Different Tools for Different Neighborhoods: Just like you wouldn't wear a heavy winter coat in the summer, the AI realized that some neighborhoods needed "Storage Tanks" (big buckets), while others just needed "Soakaways" (drainage holes). It customized the solution for every street.
• The "Middle Ground" is Safest: The AI tested different climate scenarios (mild, medium, and extreme rain). It found that planning for the medium scenario (RCP4.5) was actually the smartest bet. If you plan for the worst-case scenario, you spend too much money on defenses you might not need. If you plan for the best case, you get wiped out by a storm. The middle ground offered the best protection for the least cost.

The Analogy: The Umbrella vs. The Bunker

Imagine you live in a place where it might rain a little, a lot, or a hurricane.

• Old Way: You buy a giant, expensive bunker (extreme adaptation) just in case of a hurricane. If it only rains a little, you wasted a fortune.
• Random Way: You buy a tiny umbrella every day (random adaptation). If a hurricane hits, you get soaked.
• The AI Way: The AI is like a smart weather forecaster with a magic wallet. It watches the sky. If it sees a light drizzle coming, it buys a small umbrella. If it sees a storm brewing, it builds a temporary shelter. It knows exactly when to spend money so that you are never wet, but you never go broke buying a bunker you don't need.

Why This Matters

This paper proves that Artificial Intelligence can be a better city planner than humans when it comes to long-term, complex problems.

• It handles uncertainty: It doesn't need to know the future perfectly; it learns how to adapt as the future unfolds.
• It saves money: By finding the right balance, it prevents cities from wasting billions on unnecessary projects or losing billions to preventable floods.
• It keeps cities moving: The goal isn't just to stop water; it's to keep people getting to work, school, and the hospital, even when it's pouring rain.

The Bottom Line

Climate change is going to make our cities wetter. We can't stop the rain, but we can learn how to dance in it. This paper shows us that by using AI to play "what-if" games, we can design cities that are flexible, resilient, and ready for whatever the weather throws at us next. It's not about building a wall against the future; it's about building a city that can flow with it.

Technical Summary

1. Problem Statement

Climate change is intensifying rainfall events, leading to increased pluvial (surface) flooding in urban areas. This poses a severe threat to transportation systems, causing direct physical damage (infrastructure deterioration), direct non-physical impacts (delays, congestion), and indirect impacts (trip cancellations, accessibility loss).

Key Challenges:

• Long-term Horizon: Adaptation planning requires a 50–100 year perspective (2024–2100), far exceeding typical planning cycles.
• Deep Uncertainty: Future climate scenarios (e.g., RCP 2.6, 4.5, 8.5) are uncertain, making static optimization difficult.
• Complexity: The problem involves high-dimensional state spaces (spatially distributed zones), non-linear system dynamics, and sequential decision-making where today's investments affect future resilience.
• Trade-offs: Planners must balance immediate investment and maintenance costs against avoided future flood damages. Traditional methods often fail to capture these dynamic trade-offs or are computationally intractable for such large-scale problems.

2. Methodology

The authors propose a novel Reinforcement Learning (RL) framework integrated into an Integrated Assessment Model (IAM) to learn adaptive policies for flood mitigation in Copenhagen's inner city.

A. The Integrated Assessment Model (IAM)

The environment simulates the interaction between climate, flooding, and transport through four modules:

1. Rainfall Projection: Generates daily rainfall events for 2024–2100 based on three climate scenarios (RCP 2.6, 4.5, 8.5) using data from the Danish Meteorological Institute.
2. Flood Model: Uses SCALGO Live to simulate water accumulation and depth based on high-resolution terrain data and rainfall intensity.
3. Transport Simulation: Models 84,000 trips (car, bicycle, foot) across 29 Traffic Assignment Zones (TAZs). It calculates travel times and applies depth-disruption functions to reduce speeds based on water depth (e.g., roads become impassable at specific depths).
4. Impact Computation: Quantifies three types of costs:
   • Infrastructure Damage ($I_i$): Repair costs based on water depth and road typology.
   • Travel Delays ($D_i$): Economic value of time lost due to slower speeds.
   • Trip Cancellations ($C_i$): Value of trips abandoned due to impassable routes.

B. Reinforcement Learning Formulation

The problem is modeled as a Markov Decision Process (MDP):

• State Space ($S$): A graph representation of the city where nodes are TAZs. Features include current infrastructure damage, delays, cancellations, and the status of active interventions (e.g., remaining effectiveness of a drainage tank).
• Action Space ($A$): At each timestep, the agent selects an intervention for each of the 29 zones from a discrete set of 8 options (e.g., Do Nothing, Bioretention Planters, Soakaways, Storage Tanks, Porous Asphalt). Actions have specific lifetimes, costs, and effectiveness decay rates.
• Reward ($R$): Defined as the negative total economic cost, comprising damage costs, delay costs, cancellation costs, plus implementation and maintenance costs of interventions.
• Algorithm: The authors use Proximal Policy Optimization (PPO) with a Graph Convolutional Neural Network (GCN). The GCN allows the policy to be permutation-invariant (order-independent) and scalable to different network topologies.

3. Key Contributions

1. First Comprehensive RL Framework for Long-Term Flood Adaptation: Unlike previous studies focusing on reactive routing or short-term control, this work addresses proactive, long-term (77-year) adaptation planning for urban transport.
2. Graph-Based RL for Spatial Dynamics: The use of Graph Neural Networks (GNNs) allows the agent to learn spatial correlations between zones, enabling the discovery of coordinated adaptation pathways rather than isolated local fixes.
3. Handling Deep Uncertainty: The framework explicitly tests strategies under different "belief" scenarios (what the planner assumes) versus "realized" scenarios (what actually happens), evaluating the robustness of policies against climate uncertainty.
4. Methodological Benchmarking: The study compares RL against Bayesian Optimization (BO) on reduced problem instances, demonstrating RL's superiority in handling sequential decision-making and adaptivity.

4. Key Results

The framework was tested on Copenhagen's inner city (29 zones) over the period 2024–2100.

• Performance vs. Baselines:
  • The RL policy outperformed a "No Control" (NC) strategy by 22% and a "Random Control" (RND) strategy by 408% in cumulative reward.
  • Compared to Bayesian Optimization (BO) on reduced problems, RL achieved 0.6%–3.1% lower total costs (savings of millions of DKK). The performance gap widened with problem complexity, highlighting RL's ability to navigate the "curse of dimensionality."
• Adaptation Pathways:
  • The RL agent learned to time and spatially allocate investments. It avoided the "all-at-once" spending of the RND strategy, instead deploying measures gradually in response to evolving risks.
  • Preferred Measures: For the intermediate RCP 4.5 scenario, Soakaways (57% of actions) and Bioretention Planters (28%) were the most frequently selected, with Storage Tanks used primarily in central zones.
• Robustness to Climate Uncertainty:
  • Policies trained on the intermediate scenario (RCP 4.5) yielded the highest average reward across all three scenarios (2.6, 4.5, 8.5), suggesting it is the most cost-effective "hedging" strategy.
  • Policies trained on mild scenarios (RCP 2.6) failed significantly when tested under extreme scenarios (RCP 8.5), indicating a lack of resilience.
  • Policies trained on severe scenarios (RCP 8.5) were robust to milder realities but incurred higher precautionary costs.

5. Significance and Implications

• Decision Support: The framework serves as an exploratory decision-support system rather than a prescriptive optimizer. It helps policymakers visualize trade-offs between investment costs and avoided damages under deep uncertainty.
• Avoiding Maladaptation: By learning state-dependent policies, the system avoids "locking in" suboptimal strategies, offering flexible pathways that can adapt as climate conditions evolve.
• Scalability: The use of GNNs demonstrates that RL can scale to real-world urban networks with high-dimensional spatial decision spaces, a task where traditional dynamic programming fails.
• Policy Insight: The results suggest that a "one-size-fits-all" approach is ineffective. Instead, a coordinated mix of green infrastructure (soakaways, planters) and grey infrastructure (storage tanks), deployed strategically over time, offers the most resilient solution.

Conclusion

The paper successfully demonstrates that Reinforcement Learning is a powerful tool for long-term climate adaptation planning. By integrating complex physical models with AI, the framework discovers adaptive strategies that balance economic costs and flood resilience more effectively than traditional optimization methods, providing a robust foundation for managing climate risks in urban transportation systems.