🤖 machine learning

LakeFM: Toward a Foundation Model for Aquatic Ecosystems Using Irregular Multivariate Multi-depth Time Series Data

The paper introduces LakeFM, a foundation model pre-trained on large-scale simulated and observed ecological datasets to overcome limitations in handling irregular, heterogeneous aquatic time-series data, thereby achieving superior and physically plausible forecasting of lake dynamics and water quality.

Original authors: Abhilash Neog, Sepideh Fatemi, Medha Sawhney, Kazi Sajeed Mehrab, Aanish Pradhan, Bennett J. McAfee, Emma Marchisin, Arka Daw, Robert Ladwig, Cayelan C. Carey, Paul Hanson, Anuj Karpatne

Published 2026-06-11

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Abhilash Neog, Sepideh Fatemi, Medha Sawhney, Kazi Sajeed Mehrab, Aanish Pradhan, Bennett J. McAfee, Emma Marchisin, Arka Daw, Robert Ladwig, Cayelan C. Carey, Paul Hanson, Anuj Karpatne

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Problem: The "Messy Notebook" of Lakes

Imagine trying to understand how a lake works by reading a notebook left by a scientist. But this notebook is a mess.

The pages are missing: Some days have no entries at all.
The handwriting is different: One page measures water temperature every hour, while the next page only measures it once a month.
The depth is confusing: Sometimes they measure the surface, sometimes 5 feet down, and sometimes 20 feet down, but the depths don't match up between different lakes.
The variables are inconsistent: Lake A has data on oxygen and algae, but Lake B only has data on temperature.

For a long time, computer models (Machine Learning) struggled with this. They were like students who could only read perfectly typed, regular schedules. If the data was messy, irregular, or missing pieces, the models would break or give bad answers. They usually had to "fill in the blanks" (impute data) first, which often introduced errors.

The Solution: LakeFM (The "Super-Reader")

The authors created LakeFM, a new type of AI called a "Foundation Model." Think of it as a super-reader who doesn't need the notebook to be neat.

1. The "Token" Trick (Reading the Mess)
Instead of trying to force the messy data into a perfect grid (like a spreadsheet), LakeFM treats every single piece of information as a unique "token" or a distinct event.

Analogy: Imagine a librarian who doesn't care if books are on shelf A or shelf B, or if they are missing. They just pick up every single book (data point), read its label (what time it was, what depth, what variable), and file it in their brain.
This allows LakeFM to handle data that is irregular in time, depth, and variables without needing to "fix" or fill in the missing parts first.

2. The "Two-Brain" System
LakeFM is designed to learn two different types of things at the same time:

The "Static Brain" (The Lake's DNA): This part learns what makes a specific lake unique and unchanging, like its location, size, and whether it's fed by rain or underground springs. It's like knowing a person's fingerprint.
The "Dynamic Brain" (The Lake's Mood): This part learns how the lake changes over time, like how the temperature shifts with the seasons or how algae blooms in the summer. It's like tracking a person's daily mood swings.
By separating these, the model can recognize that a lake in Wisconsin is different from a lake in Florida, even if they are both cold in the winter.

3. The "Universal Translator"
LakeFM was trained on a massive library of data:

Real-world data: Observations from 21 actual lakes in the US (which are very messy and sparse).
Simulated data: Over 1,000 computer-generated lakes based on physics laws.
Because it learned from both real messiness and perfect physics, it can now look at a lake it has never seen before (Zero-Shot) and make good predictions. It's like a chef who has cooked in 20 different kitchens and can now walk into a brand new kitchen and cook a great meal using whatever ingredients are available.

What Can LakeFM Do?

1. Predicting the Future (Forecasting)
The paper shows that LakeFM can predict future water conditions (like temperature or oxygen levels) better than existing models, even when the data is missing or irregular.

The Result: It beats other top AI models (like Chronos 2 or MOMENT) in accuracy, especially when looking at lakes it has never seen before.

2. The "What-If" Detective
One of the coolest features is that LakeFM can handle "masked" data. You can tell it, "I only have temperature data, no oxygen data," and it can still predict the oxygen levels by using what it learned about how temperature and oxygen interact in other lakes.

The Insight: The paper found that if you hide the oxygen data, the model gets more uncertain (which is smart!), but if you hide the temperature, the model gets confused. This helps scientists understand which variables are most important for predicting others.

3. Following the Laws of Physics
Even though LakeFM is just an AI, it learned to respect the laws of nature.

Thermal Stratification: In summer, lake water gets colder as you go deeper. LakeFM rarely makes mistakes where it predicts the deep water is hotter than the surface (a physical impossibility).
Light and Algae: It correctly predicts that light fades as you go deeper, just like real physics says it should.
The Metaphor: It's like a student who didn't just memorize the answers but actually understood the rules of the game, so they don't make "illegal moves."

4. Understanding Lake Personalities
The paper visualized the AI's "thoughts" (embeddings) and found that it naturally groups lakes together based on their real-world characteristics.

Lakes that are geographically close or have similar water types (like "drainage" vs. "seepage" lakes) ended up next to each other in the AI's mental map.
This proves the AI isn't just guessing; it has learned the actual "personality" of different lake ecosystems.

Summary

LakeFM is a new AI tool that acts like a master detective for lakes. It can read messy, incomplete, and irregular data from any lake, understand the unique "personality" of that lake, and predict its future behavior while respecting the laws of physics. It doesn't need perfect data to work, making it a powerful tool for understanding and protecting our freshwater ecosystems.

Technical Summary: LakeFM: Toward a Foundation Model for Aquatic Ecosystems

1. Problem Statement

Monitoring the health of inland water bodies is critical for sustainable freshwater management, yet modeling lake dynamics faces significant challenges due to the nature of ecological data. Existing machine learning approaches for ecological time-series data typically assume regular sampling in both time and depth. However, real-world lake monitoring data is characterized by:

Irregularity: Highly variable sampling frequencies (from daily to monthly) and missing values (60–70% sparsity on average in datasets like LakeBeD-US).
Heterogeneity: Diverse sets of variables, varying depth resolutions, and inconsistent observation patterns across different lakes.
Scalability Issues: Current Time Series Foundation Models (TSFMs) like Chronos 2 or Moment generally assume clean, densely sampled, or univariate data, making them ill-equipped for the sparse, multivariate, multi-depth nature of aquatic science. Furthermore, existing models often struggle to generalize across lakes with different physical and biogeochemical characteristics without extensive per-lake fine-tuning.

The core problem addressed is the lack of a foundation model capable of learning generic lake processes across a broad collection of lakes while retaining site-specific nuances, specifically handling irregular, multivariate, multi-depth time series data without relying on external imputation.

2. Methodology: LakeFM Architecture

LakeFM is a foundation model pre-trained on a large-scale dataset comprising over 1.5 million samples from 21 real-world lakes (LakeBeD-US) and over 1,000 diverse process-based simulations (WQHanson and FCR simulations). The architecture is designed to operate on an irregular grid, treating data as a sequence of events rather than a fixed grid.

2.1 Input Tokenization and Embedding

Unlike grid-based approaches, LakeFM tokenizes every individual measurement as a distinct event (token).

Token Definition: Each observation is a tuple $(t_k, v_k, d_k, x_k)$ , representing absolute timestamp, variable identifier, continuous depth, and scalar value.
Embedding Strategy: A composite embedding is constructed by concatenating four components:
- Time Embedding: Sinusoidal positional encodings for continuous time.
- Depth Embedding: Fourier feature encoding to handle continuous depth values, allowing the model to learn vertical gradients.
- Variate Embedding: Learnable lookup for categorical variable IDs.
- Value Embedding: A SwiGLU-style gated projection for the scalar measurement value.
Sequence Formation: Tokens are sorted by variable ID, then depth, and finally by absolute time to form the input sequence.

2.2 Encoder and Feature Disentanglement

The encoder utilizes a stack of Transformer blocks with specific modifications for scientific data:

Rotary Position Embeddings (RoPE): Used to model relative temporal dependencies.
Structured Attention Bias: A learnable additive bias is introduced to attention logits to differentiate between intra-variate (temporal dynamics of a single variable) and inter-variate (correlations between different variables) interactions.
Static vs. Dynamic Disentanglement: The model decouples the representation space into two parallel subspaces:
- Static Embeddings: Aggregated via mean pooling to capture time-invariant lake characteristics (e.g., geography, hydrology). These are used for contrastive learning.
- Temporal Embeddings: Retain temporal information and serve as input for the forecasting decoder.

2.3 Query-Based Forecasting and Probabilistic Output

Query-Based Strategy: Instead of a fixed feed-forward decoder, LakeFM uses target queries defined by specific $(t, v, d)$ points. The decoder employs cross-attention to retrieve relevant information from the encoded historical context, enabling flexible inference at arbitrary unobserved points (regular or irregular grids).
Probabilistic Head: To handle heavy-tailed noise and uncertainty common in environmental data, the model outputs parameters for a Student- $t$ distribution ( $\mu, \sigma, \nu$ ). Crucially, the degrees of freedom ( $\nu$ ) are learned dynamically, allowing the model to adapt between heavy-tailed and Gaussian-like regimes based on the specific variable and context.

2.4 Training Objectives

LakeFM is pre-trained using a joint optimization of two losses:

Probabilistic Forecasting Loss: Minimizes the negative log-likelihood of ground truth values under the predicted Student- $t$ distribution.
Contrastive Loss: Encourages the model to learn lake-specific static representations by treating samples from the same lake as positives and different lakes as negatives (InfoNCE formulation).

3. Key Contributions

Irregular Data Handling: LakeFM is the first foundation model designed to ingest irregular, multivariate, multi-depth time series data directly, eliminating the need for imputation or resampling to regular grids.
Zero-Shot Generalization: The model demonstrates competitive or superior forecasting performance on unseen lakes (Out-of-Distribution) and unseen variables, leveraging knowledge transfer across diverse lake systems.
Physical Consistency: The model exhibits emergent adherence to physical laws (e.g., thermal stratification and Beer-Lambert law) without explicit physical constraints in the loss function.
Scientific Discovery: Through masking experiments, LakeFM reveals novel insights into variable interactions and depth dependencies, acting as a discovery engine for ecological hypotheses.
Efficiency: Despite handling complex irregular data, LakeFM achieves these results with significantly fewer parameters (7M) compared to general-purpose TSFMs.

4. Experimental Results

The model was evaluated on the LakeBeD-US dataset (21 lakes) and simulation datasets, comparing against Time Series Foundation Models (Chronos 2, MOMENT, MOIRAI), local models (iTransformer), and irregular time-series models (HyperIMTS, ReIMTS).

Forecasting Performance: LakeFM achieved the lowest Mean Squared Error (MSE) across most In-Distribution (ID) and Out-of-Distribution (OOD) lakes. It consistently outperformed baselines, particularly in zero-shot settings where other models struggled due to a lack of transfer learning capabilities.
Imputation-Free Advantage: In comparisons with irregular time-series baselines, LakeFM achieved the best average rank (1.0 in OOD, 1.27 in ID), confirming that its performance gains stem from direct modeling of irregularities rather than just avoiding imputation pipelines.
Physical Consistency: On 100 unseen simulated lakes, LakeFM showed higher consistency with the Thermal Stratification Law (lower inversion rates) and the Beer-Lambert Law (higher $R^2$ ) compared to Chronos 2.
Embedding Analysis:
- Static Embeddings: t-SNE visualizations showed that learned static embeddings naturally clustered lakes by geographic location (State) and hydrologic regime (e.g., drainage vs. seepage), despite these metadata not being used during training.
- Dynamic Embeddings: Temporal trajectories in the embedding space aligned with ecological characteristics, such as trophic state (oligotrophic vs. eutrophic) and seasonal patterns.
Ablation Studies: Removing contrastive learning, variate-specific degrees of freedom, or continuous depth embeddings resulted in significant performance degradation, validating the necessity of each component.

5. Significance and Claims

The paper positions LakeFM as a practical step toward scalable and generalizable modeling of lake ecosystems. Its significance lies in:

Paradigm Shift: Moving away from single-lake, single-variate analyses and imputation-dependent pipelines toward a unified foundation model that shares information across disparate lakes with varying irregularities.
Macro-System Understanding: Enabling the first step toward understanding lake ecology at a macro-system scale using diverse and heterogeneous data.
Scientific Utility: The ability to handle masked variables allows researchers to query "what-if" scenarios (e.g., predicting temperature without dissolved oxygen data), revealing insights into variable interactions that can guide future data collection strategies.
Foundation for AI4Science: The work demonstrates that foundation models can be tailored to specific scientific domains, learning not just statistical patterns but also underlying physical principles, bridging the gap between data-driven ML and process-based scientific modeling.

The authors acknowledge limitations regarding extrapolation to climatic contexts significantly different from the training distribution (e.g., regions with different thermal/ice-cover dynamics), noting that the model relies on learned statistical dependencies within the observed data range.