🤖 AI

Physics-Guided Spatiotemporal Learning for Coastal Wave Peak Period Estimation from Video

This paper proposes a physics-guided deep spatiotemporal learning framework that leverages passive coastal video streams, Sim-to-Real transfer learning, and physics-informed regularization to accurately and consistently estimate nearshore wave peak periods, offering a cost-effective and operationally feasible alternative to traditional monitoring systems.

Original authors: Abubakar Hamisu Kamagata, Dharm Singh Jat, Attlee Munyaradzi Gamundani, Abhishek Srivastava, Paramasivam Saravanakumar

Published 2026-06-12

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Abubakar Hamisu Kamagata, Dharm Singh Jat, Attlee Munyaradzi Gamundani, Abhishek Srivastava, Paramasivam Saravanakumar

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to guess the rhythm of a song just by watching a video of people dancing. Usually, you might need a microphone to hear the beat clearly. But what if you could figure out the exact tempo just by watching the dancers' movements, even if the video is shaky, the lighting is bad, or the crowd is noisy?

That is essentially what this paper does, but instead of dancers, it's watching ocean waves, and instead of a song's tempo, it's calculating the wave peak period (how long it takes for a wave to travel from one crest to the next). This information is vital for protecting coastlines, but traditional ways to get it (like buoys or radar) are expensive and hard to cover everywhere.

Here is how the researchers built a "smart video watcher" to solve this problem, explained in simple terms:

1. The Problem: Too Much Noise, Not Enough Data

Ocean videos are messy. They have sand, sky, clouds, and camera shakes. Also, there aren't many "expert-labeled" videos where a human has already told the computer exactly what the wave period is. It's like trying to learn a language when you only have a few dictionaries and a lot of confusing radio static.

2. The Solution: A Three-Step "Training Camp"

The researchers created a system that trains a computer in three distinct phases, moving from fake data to real data, and finally to expert data.

Phase 0: The "Virtual Reality" Gym (Synthetic Pre-training)
Before showing the computer real ocean videos, they fed it thousands of fake, computer-generated waves based on simple physics rules (called Airy Wave Theory).
- Analogy: Imagine a pilot learning to fly in a flight simulator before ever touching a real plane. The computer learns the basic "rules of the road" for how waves move without getting confused by real-world messiness.
Phase 1: The "Noisy" Classroom (Silver Set)
Next, they showed the computer real videos from the internet (like stock footage). These videos didn't have perfect labels, so the computer had to guess the wave periods based on how the pixels moved (like optical flow).
- Analogy: This is like a student listening to a radio broadcast with static. They aren't getting perfect information, but they are learning to recognize the general shape and rhythm of the waves in the real world.
Phase 2: The "Expert" Tutor (Golden Set)
Finally, they showed the computer a small, high-quality set of videos filmed on the coast of Namibia. These were carefully checked and labeled by human experts. The computer fine-tuned its knowledge here to get the details right.
- Analogy: This is the final exam with a strict teacher who corrects every tiny mistake, ensuring the student is ready for the real world.

3. The Secret Sauce: "Physics Rules"

One of the biggest innovations is that they didn't just let the computer guess freely. They built a "guardrail" into the system.

The Guardrail: The computer knows that waves in this part of the ocean can't be shorter than 2 seconds or longer than 20 seconds (based on real physics). If the computer tries to guess a 1-second wave or a 100-second wave, the system automatically says, "No, that's impossible," and corrects it.
Analogy: It's like a spell-checker that doesn't just check spelling, but also checks if the sentence makes sense physically. If you type "The sun rose in the west," it might let the spelling pass, but this system would say, "Wait, the sun doesn't do that."

4. Focusing the Eyes: The "Active Zone" Detector

The system doesn't look at the whole video frame (which includes the sky and the beach). It automatically finds the "active zone" where the water is actually moving and ignores everything else.

Analogy: Imagine trying to hear a conversation in a crowded room. Instead of listening to the whole room, you put on noise-canceling headphones that only let you hear the two people talking. This system does that for waves, ignoring the sand and sky.

5. The Results: Two Different "Winners"

The researchers tested different types of computer brains (architectures) and found two different champions, depending on what you need:

The "Instant Snapshot" Champion (LtViViT):
This model is like a super-fast photographer. It is incredibly accurate at guessing the wave period for a single moment in time. If you need to know right now what the wave is doing, this is the best choice.
The "Steady Trend" Champion (TinyWaveNet with Physics Rules):
This model is like a wise old sailor. It might be slightly less perfect at a single snapshot, but it is much better at predicting the overall trend over time. It rarely makes wild, impossible mistakes because of the "physics guardrails."
- Why this matters: For long-term coastal monitoring (like checking if a beach is eroding over a year), you want the "Steady Trend" model because it is reliable, consistent, and doesn't get distracted by noise.

6. Why This is a Big Deal

Cost: It turns a simple camera (which costs very little) into a scientific instrument.
Speed: The best model is so lightweight it can run on small, cheap devices (like a Raspberry Pi) right at the beach, without needing a supercomputer.
Reliability: By combining deep learning with actual physics rules, the system doesn't just "guess"; it understands the laws of nature.

In summary: The paper presents a smart, cheap, and reliable way to watch ocean waves using video. It teaches a computer to ignore the noise, learn from fake waves first, and then use real-world physics rules to ensure its guesses are always scientifically valid. This helps coastal managers protect shorelines without needing expensive, high-maintenance equipment.

Technical Summary: Physics-Guided Spatiotemporal Learning for Coastal Wave Peak Period Estimation from Video

Problem Statement
Accurate monitoring of nearshore wave dynamics, specifically the wave peak period ( $T_p$ ), is essential for coastal engineering, hazard mitigation, and climate resilience. Traditional in-situ systems (buoys, LiDAR) and remote sensing platforms offer accuracy but suffer from high deployment costs, maintenance difficulties, and limited spatial coverage. While video-based monitoring has emerged as a cost-effective alternative, existing deep learning approaches often lack physical interpretability, rely on intermediate preprocessing steps (e.g., timestack construction, spectral analysis), or fail to generalize across diverse coastal morphologies due to a scarcity of expert-annotated datasets. There is a critical need for an end-to-end deep learning framework that directly estimates $T_p$ from raw monocular video while adhering to hydrodynamic physical constraints.

Methodology
The authors propose a Physics-Guided Deep Spatiotemporal Learning Framework designed to estimate nearshore wave peak periods directly from raw video streams. The methodology is structured around three core components:

Automated Region of Interest (ROI) Detection:
Instead of manual cropping, the system employs a temporal-variance-based detector. By analyzing pixel intensity variance over time, the algorithm isolates the "active band" of the surf zone, filtering out static land and sky regions. This process is architecture-agnostic, ensuring consistent input focusing across different model backbones.
Tri-Phase Transfer Learning Strategy:
To address data scarcity, the framework utilizes a three-stage training pipeline:
- Phase 0 (Synthetic Pre-training): Models are pre-trained on over 1,000 synthetic video sequences generated using Linear Airy Wave Theory. This instills fundamental wave propagation physics and periodic relationships before exposure to real-world noise.
- Phase 1 (Silver Set Pre-training): Models are trained on a larger dataset of real coastal videos (20 scenes) with "noisy" labels generated via an optical flow-based pseudo-labelling pipeline. This stage adapts the models to real-world lighting, texture, and environmental variations.
- Phase 2 (Golden Set Fine-tuning): The models are fine-tuned on a high-quality, expert-annotated dataset (13 scenes, ~6,900 clips) from the Namibian coast. This stage aligns the model predictions with ground truth derived from TimeStack analysis and expert visual assessment.
Physics-Guided Loss Function:
The training objective combines a data fidelity term (Smooth Huber loss) with a physics-informed regularization term. This penalty discourages predictions outside the physically plausible range for deep-to-intermediate water waves (8–20 seconds, corresponding to 0.05–0.125 Hz), ensuring the model respects hydrodynamic constraints defined by Airy wave theory.
Model Architectures:
The study evaluates several spatiotemporal architectures, including:
- LtViViT: A lightweight Video Vision Transformer.
- PtAttnCNN (TinyWaveNet): A ResNet18-based architecture with attention mechanisms.
- Recurrent Models: PtLSTM and WaveConvLSTM.

Key Results
The evaluation, conducted on the Namibian coast under varying conditions (open beaches, rocky headlands, harbors), yielded the following findings:

Architecture Performance Trade-offs:
- LtViViT achieved the highest instantaneous prediction accuracy with the lowest Root Mean Square Error (RMSE) of 0.7692 s in Phase 2. It excels at capturing localized spatiotemporal dependencies.
- PtAttnCNN (TinyWaveNet) demonstrated superior oceanographic skill, achieving the best Willmott Skill Score (WS: 0.9811 with $\lambda=5.0$ ) and the lowest Scatter Index (SI: 0.0892). While its RMSE was higher (1.4898 s), it exhibited tighter trend-following consistency and better overall operational reliability.
Impact of Physics-Guided Regularization:
Ablation studies confirmed that increasing the physics weight ( $\lambda$ ) to 5.0 significantly improved the Willmott Skill Score and reduced the Scatter Index. The physics constraint effectively suppressed physically implausible predictions (e.g., periods < 2s) and reduced spurious spectral peaks caused by visual artifacts like camera shake.
Efficiency and Feasibility:
The proposed TinyWaveNet + $\lambda=5.0$ configuration is ultra-lightweight (0.28 M parameters) with an inference latency of 0.53 ms, making it suitable for real-time edge deployment.
Physical Consistency Auditing:
Post-hoc auditing using Grad-CAM revealed that the model directs 61.28% of its spatial attention to hydrodynamically active surf zones. Furthermore, a strong Pearson correlation (R = 0.842) was found between predicted $T_p$ and physically derived pixel speeds, validating that the model learned the dispersion relationship rather than image artifacts.

Significance and Claims
The paper claims that this framework represents a significant step toward cost-efficient, physically consistent, and operationally feasible coastal monitoring. By integrating automated ROI detection, multi-stage transfer learning (Sim-to-Real), and physics-informed regularization, the system overcomes the limitations of traditional video processing (sensitivity to lighting and calibration) and pure data-driven models (lack of physical grounding).

The authors emphasize that the study highlights a crucial trade-off: while Transformer-based models (LtViViT) offer superior point-to-point accuracy, lightweight recurrent-convolutional models with strong physics regularization (PtAttnCNN) provide better long-term trend consistency and operational skill. The framework is presented as a viable alternative to expensive sensor arrays for long-term monitoring of wave peak periods, supporting coastal management, erosion control, and climate adaptation strategies. The authors note that while the current validation is limited to the South Atlantic/Namibian context due to dataset size, the methodology is designed to be scalable to other wave climates in future work.