Technical Summary: HydroAgent

Problem Statement

Calibrating distributed hydrologic models is a critical bottleneck in operational water-resources management, underpinning tasks ranging from flash-flood forecasting to reservoir operation. Currently, this process relies on human domain experts who manually translate hydrograph signatures into adjustments of high-dimensional parameter vectors. This workflow is non-scalable: it requires hours to days of expert attention per basin, and knowledge gained on one watershed does not transfer effectively to another. As climate change accelerates the hydrologic cycle, the demand for accurate, basin-specific simulations is rising faster than the supply of trained hydrologists.

The paper investigates whether frontier Large Language Model (LLM) agents can replace human hydrologic modelers. While agents possess the reasoning capacity to read data, edit control files, and invoke simulators, it remains unclear if they can perform the sustained, iterative refinement required for effective calibration.

Methodology

1. Benchmarking Frontier Agents

The authors first established a baseline by evaluating nine frontier LLM agents (including Claude Opus 4.6/4.7, Sonnet 4.6, GPT-5 variants, and Gemini 2.5/3.1/3-flash) on the calibration of the CREST (Coupled Routing and Excess STorage) distributed hydrologic model.

• Environment: Agents operated within a Linux sandbox using the EF5 (Ensemble Framework For Flash Flood Forecasting) simulator.
• Task: Agents were given basin descriptors, gauge time series, and raster forcings. Their goal was to adjust 13 scalar multipliers (e.g., soil-water capacity, infiltration shape, channel-routing coefficients) to maximize the Nash–Sutcliffe Efficiency (NSE) between simulated and observed streamflow.
• Constraints: Each agent had a fixed compute budget of 20 rounds (up to 200 simulations) and a 2-hour timeout. The evaluation covered four held-out gauges spanning diverse hydroclimatic regions and basin sizes (329–40,792 km²).

2. The HydroAgent Framework

To address the limitations observed in frontier agents, the authors propose HydroAgent, a domain-specific agent built on the open-weight Qwen3-4B-Instruct model. The training follows a two-stage recipe:

• Stage 1: Supervised Fine-Tuning (SFT):

  • The base model was fine-tuned on 2,576 expert calibration trajectories distilled from a stronger proprietary teacher (GPT-5).
  • These trajectories preserved the full iterative sequence of hypothesize → simulate → diagnose → adjust, teaching the model the tool-call grammar and basic hydrologic reasoning (e.g., linking a steep recession limb to specific parameter adjustments).
  • Trajectories with final NSE < 0.6 were discarded.

• Stage 2: Reinforcement Learning with Simulation Feedback (RLSF):

  • The SFT-initialized model underwent reinforcement learning using Group-Relative Policy Optimization (GRPO).
  • Verifier: Unlike human-feedback RL, the reward signal came directly from the CREST simulator.
  • Reward Structure: The reward function combined a terminal score (clipped NSE) with per-turn shaping signals. Crucially, it included bonuses for valid parameter sets, successful simulations, and, most importantly, improvements in NSE over previous turns. This explicitly trained the agent to "iterate until it cannot improve," mimicking human expert behavior.
  • Infrastructure: Training utilized 4× NVIDIA H100 GPUs with the verl framework and SGLang for multi-turn rollouts.

Key Results

Frontier Agent Performance

• Performance Ceiling: The best frontier agents (Sonnet 4.6, Opus 4.7, Gemini 3-flash) achieved a mean best-of-twenty-rounds NSE in the 0.65–0.75 range.
• Human Gap: Only Opus 4.7 reached the human-expert reference (NSE ≈ 0.85) on a single test gauge. Most models failed to reach the "good" performance band (NSE > 0.70) consistently.
• Failure Modes:
  1. Premature Termination: Pro-tier reasoning models often stopped after 1–2 rounds, deeming the result "acceptable" despite remaining budget.
  2. Out-of-Bounds Proposals: Weaker models frequently suggested physically impossible parameter values (e.g., negative impervious area).
  3. Diagnosis-Action Mismatch: Agents often correctly identified hydrograph issues (e.g., recession shape) but proposed adjustments to the wrong physical parameters (e.g., changing routing instead of soil moisture).

HydroAgent Performance

• Metric Improvement: HydroAgent significantly improved calibration metrics across all four held-out basins compared to the untuned Qwen3-4B base.
  • NSE: Improved by +5% to over +200% (e.g., shifting from deeply negative to positive on gauge 01403060).
  • Correlation & Timing: Discharge correlation improved by up to 41%, and timing offsets contracted significantly.
• Ablation Insights:
  • SFT Alone: Surprisingly, SFT-only models performed worse than the untuned base on most gauges. They learned the tool format but lost the iterative engagement, often terminating after a single guess.
  • RLSF Necessity: The addition of RLSF was critical. It restored the long-horizon engagement, forcing the model to utilize the full 50-turn budget and iteratively refine parameters based on simulator feedback.
  • Synergy: The combination of SFT (providing domain vocabulary and tool grammar) and RLSF (providing the incentive for iterative refinement) was required to close the gap with frontier models.

Significance and Claims

The paper argues that the gap between frontier LLMs and human experts in hydrologic calibration is not a parameter-count problem but a domain-grounding problem.

1. Efficiency over Scale: For Earth-system tasks involving cheap-to-evaluate physical simulators, a small, domain-tuned policy (4B parameters) combined with simulator-in-the-loop RL is a more compute-efficient and physically faithful path than scaling generic frontier models.
2. The Role of Multi-Modal Data: Earth-system science offers a unique opportunity for AI agents because of the richness of multi-modal data (remote sensing, in-situ time series, and forecaster narratives). A domain agent trained to fuse this data with a physical solver can yield steerable, physically consistent predictions.
3. Reproducibility: The authors provide a full recipe for reproducing the training (SFT + GRPO) and benchmarking, including the release of 180 calibration trajectories (9 agents × 20 rounds) and full training configurations.

The authors conclude that while frontier agents are within striking distance of human performance, they lack the specific grounding to close the final gap. HydroAgent demonstrates that a specialized, simulator-grounded approach can substantially narrow this gap without requiring massive generalist models.