⚡ electrical engineering

Do AI dream of P and S picks? Multimodal LLMs applied to visual seismological analysis

This study demonstrates the feasibility of applying multimodal Vision-Language Models to visual seismological analysis by introducing a novel foveated vision framework and prompt system that replicates expert heuristics for pattern recognition while addressing critical failure modes through a morphological abstention protocol, ultimately enabling transparent and sovereign local AI deployments in geosciences.

Original authors: Raffaele Di Stefano, Valentino Lauciani, Fabrizio Bernardi, Daniele Bailo, Maddalena Michele, Maria Grazia Ciaccio, Simone Marzorati

Published 2026-09-21

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Raffaele Di Stefano, Valentino Lauciani, Fabrizio Bernardi, Daniele Bailo, Maddalena Michele, Maria Grazia Ciaccio, Simone Marzorati

Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

In the quiet hum of a seismology lab, the work of finding an earthquake begins with a look. When the ground shakes, sensors record the motion as a squiggly line on a graph, a visual story of energy traveling through the earth. For decades, experts have studied these lines by eye, scanning for the exact moment a wave begins, a skill that relies on recognizing patterns, comparing the shape of the signal to the noise around it, and understanding the context of the entire record. This visual intuition is difficult to teach to a computer, which traditionally sees data only as rows of numbers. While modern software can process vast amounts of seismic data quickly, it often struggles to replicate the flexible, context-aware way a human brain spots a faint tremor hidden in static. The question researchers asked was whether a new kind of artificial intelligence, one that can see images and read text simultaneously, could learn to look at these graphs the way a human expert does.

A team of scientists at the Istituto Nazionale di Geofisica e Vulcanologia in Italy set out to answer this by teaching a multimodal artificial intelligence to act as a seismologist. They did not simply feed the computer raw numbers; instead, they showed it pictures of the seismic waves, just as a human analyst would. The researchers built a system that mimics the way a human eye focuses. First, the AI looks at a wide view of the entire recording to get a general sense of where the waves might be. Then, it zooms in on those specific areas, magnifying the details to pinpoint the exact moment the wave started. This process, which the team calls an adaptive foveated workflow, allows the model to shift its attention from a broad overview to a microscopic inspection, much like a person squinting at a map to read a small label.

The results showed that this visual approach worked remarkably well. When the AI examined the full, wide view of the seismic data, it could identify the start of the waves with reasonable accuracy, but it often missed the precise millisecond timing. However, when the system zoomed in on the specific moments of interest, the accuracy improved dramatically. The model learned to distinguish the sharp, sudden start of a primary wave from the background noise with a precision that matched human experts. In tests involving dozens of recording stations and earthquakes of varying sizes, the system successfully identified the arrival times of the waves within a fraction of a second. The researchers found that the quality of the image mattered as much as the intelligence of the model; using clear, uncompressed pictures of the waves allowed the AI to see fine details that were lost in blurry or compressed versions.

Despite these successes, the researchers discovered a significant flaw in how these artificial minds operate. The AI has a strong tendency to follow instructions, even when the data is too messy to provide a clear answer. If asked to find a wave, the model would sometimes invent a time and location just to satisfy the request, even if the graph showed only noise. This behavior, which the team calls compliance bias, meant the system could confidently report a fake earthquake. To fix this, the scientists proposed a new safety rule for the AI: a "surrender clause." This rule instructs the model to admit when it cannot see a wave clearly and to stop, rather than guessing. By allowing the AI to say "I don't know" when the signal is corrupted, the system becomes more reliable and safer to use in real-world monitoring.

The study also compared running this system on powerful commercial cloud servers versus running it on local, open-source computers. The cloud version performed better, largely because it could handle high-resolution images without losing detail. The local version struggled more, often blurring the fine lines of the graph, which made it harder to spot the exact start of a wave. This finding suggests that while it is possible to run these advanced tools on local machines, the quality of the visual input is critical. The researchers concluded that for this technology to be truly useful in science, it must be able to see the data with the same clarity as a human, and it must be smart enough to know when the data is too poor to trust.

Ultimately, this work demonstrates that artificial intelligence can learn to interpret complex scientific graphics not just by crunching numbers, but by seeing patterns. The system does not replace the human expert but acts as a powerful assistant that can scan thousands of records with the same visual logic a human uses. By combining a zooming visual strategy with strict rules that prevent the AI from guessing, the researchers have created a proof-of-concept for a new kind of seismic monitoring. It is a step toward a future where machines can help scientists understand the earth's movements with greater speed and transparency, provided they are taught not only how to look, but also when to stop.

Technical Summary: Multimodal LLMs Applied to Visual Seismological Analysis

1. Problem Statement

Conventional deep learning pipelines for seismic processing, while efficient at high-throughput numerical analysis, often fail to replicate the contextual visual reasoning inherent to human seismologists. Identifying P- and S-wave onsets is not merely a numerical calculation but a comparative, visual process dependent on spatiotemporal context, noise suppression, and geometric pattern recognition. Current automated methods frequently reduce complex perceptual tasks to rigid matrices, lacking the "visual-cognitive" flexibility required to handle diverse signal-to-noise ratios and attenuated waveforms. Furthermore, standard Vision-Language Models (VLMs) face specific bottlenecks when applied to seismograms, including attention saturation from excessive text, resolution mismatches that degrade high-frequency signal features, and a "compliance bias" where models generate speculative numerical picks even when visual evidence is insufficient.

2. Methodology

The study introduces SAIAN (Seismological AI ANalysis), a proof-of-concept framework leveraging Multimodal Large Language Models (VLMs) to emulate expert human visual inspection. The methodology is built on three core pillars:

A. Adaptive Foveated Vision Framework

To overcome the resolution limits of standard Vision Transformers (ViT), which process images in fixed $16 \times 16$ pixel patches, the authors implemented a multi-stage "glance-and-focus" workflow:

Stage 1 (Global Overview): The model analyzes a high-resolution, lossless PNG canvas containing three-component (Z, N, E) waveforms. This provides a holistic view to identify candidate arrival times.
Stage 2 (Adaptive Zoom): Based on Stage 1 outputs, a Python orchestrator dynamically crops the raw NumPy array around the candidate timestamps. These narrow windows are re-projected onto the same fixed image canvas dimensions, effectively increasing the pixel-per-second density. This forces the ViT's attention mechanism to focus on the micro-structure of the waveform breakout.
StageMax (Composite Refinement): Rather than applying a uniform third stage to all stations, an ultra-fine zoom (Stage 3, $\pm 0.4$ s window) was applied selectively only to stations where Stage 2 residuals were $\ge 0.2$ s. This composite "StageMax" tier represents the highest resolution attained per station, combining targeted ultra-fine zooms with standard Stage 2 context outputs to achieve millisecond-level precision where needed.

B. Input Fidelity and Format Optimization

The study rigorously evaluates input formats, identifying PNG (lossless DEFLATE) as the superior format for VLM ingestion.

JPEG introduces ringing and blocking artifacts that corrupt patch embeddings.
PDF/SVG suffer from server-side downsampling or context window saturation due to massive token counts.
PNG preserves bit-perfect geometric fidelity, ensuring the model's self-attention mechanisms compute weights on physical signal features rather than compression noise.

C. Prompt Engineering and Cognitive Offloading

To mitigate attention saturation and "compliance bias" (the tendency of models to force an answer even when uncertain), the authors employed:

Cognitive Offloading: Stripping the system prompt of dense theoretical literature to free the context window for visual tokens.
Markdown Scaffolding: Using structural headers and bold syntax to anchor attention to critical operational directives.
Morphological Abstention Protocol (Proposed): To address the decoupling between the model's qualitative diagnostics and its quantitative outputs, the authors propose a conceptual two-tier framework. This includes a "Surrender Clause" where, if a waveform exhibits un-filterable corruption, the model is instructed to set numerical parameters to null. The paper notes that the full operational integration and validation of this closed-loop abstention protocol are designated for future study.

3. Key Results

The framework was benchmarked against 11 local earthquakes (magnitude 2.0–3.1) across the Italian national network, comparing AI picks to manual expert picks.

Performance Convergence via Zooming

Stage 1 (Global): High-precision picks ( $\le \pm 0.10$ s) were limited (17.8% for P-waves, 12.6% for S-waves) due to spatial compression.
Stage 2 (Zoom): High-precision accuracy more than doubled for P-waves (40.4%) and tripled for S-waves (41.0%). Gross temporal outliers ( $> 0.50$ s) were reduced by more than half.
StageMax (Composite): By applying ultra-fine zooms selectively to stations with elevated residuals, P-wave accuracy within $\le \pm 0.10$ s reached 67.6%, with 42.9% achieving sub-50 ms precision. S-wave accuracy reached 52.9% within $\le \pm 0.10$ s.
Distance Independence: The foveated zoom successfully counteracted the performance degradation typically seen at far-field distances (60–100 km), where crustal attenuation usually smears phase onsets.

Automated vs. Interactive Performance

API Execution: Direct API calls (Google Gemini 1.6 Flash) outperformed the interactive web interface (WebUI) because they bypassed client-side browser downsampling and lossy compression, delivering pristine PNG rasters directly to the vision encoder.
Local Deployment: Self-hosted open-weight models (Gemma via Ollama) showed significantly lower performance due to fixed-resolution downsampling and weight quantization, highlighting the current gap between commercial cloud vision encoders and local open-source alternatives.

Operational Robustness

The automated machine-to-machine (M2M) orchestrator successfully processed 91 stations in under 2 hours with a low protocol violation rate (2.7%). The system demonstrated "graceful degradation," halting refinement loops for specific stations that failed JSON schema validation while continuing to process the rest of the network.

4. Significance and Claims

The paper positions this work as a proof-of-concept demonstrating that multimodal generative AI can replicate expert visual heuristics in seismology, moving beyond rigid numerical processing to flexible, context-aware interpretation.

Visual Advantage: The study claims that the primary driver of accuracy is not massive computational overhead, but the structural alignment of visual inputs (lossless PNGs) and adaptive scaling (foveated zooming) with the model's attention mechanisms.
Scientific Sovereignty: It establishes the feasibility of local, in-house AI deployments for geosciences, laying the foundation for transparent, reproducible, and institutionally sovereign AI ecosystems, provided that local vision encoders are advanced to support dynamic high-resolution tiling.
Behavioral Alignment: The authors identify "compliance bias" as a critical failure mode in current VLMs for scientific tasks. They propose that future operational pipelines must integrate morphological abstention protocols to prevent the model from generating false-positive numerical data when visual evidence is ambiguous, though full implementation remains a subject for future research.

The paper concludes that while the current system requires human-in-the-loop validation or specific guardrails for full autonomy, the "Adaptive Foveated Loop" successfully bridges the gap between human visual expertise and machine inference, offering a new paradigm for automated seismic phase picking.