Toward a Scientific Discovery Engine for Weather and Climate Data: A Visual Analytics Workbench for Embedding-Based Exploration

This paper presents an open-source visual analytics workbench that enables scientists to interpret, validate, and explore embedding-based representations of large-scale weather and climate data by linking latent-space search results back to their physical origins and metadata, thereby facilitating a discovery workflow for identifying and retrieving analog events like tropical cyclones.

The Gist

Imagine you are a weather detective trying to solve a mystery. You have a library containing petabytes of data—essentially, every single weather map, wind speed chart, and temperature reading generated by supercomputers and AI models for years. It's so much information that no human could ever read it all, let alone find a specific pattern hidden inside.

This paper introduces a new "Scientific Discovery Engine" (a visual workbench) designed to help scientists navigate this massive library. Here is how it works, explained simply:

1. The Problem: The "Black Box" of AI Search

Scientists are starting to use AI to turn complex weather maps into mathematical "fingerprints" (called embeddings).

• The Analogy: Imagine turning a photo of a hurricane into a long list of numbers. If two hurricanes look similar, their number-lists will be close together in a giant mathematical space.
• The Catch: Just because two number-lists are close together doesn't mean the weather is actually similar. They might be close just because of how the computer processed the data, or because they happened in the same country, or because of a glitch in the model.
• The Risk: If a scientist trusts the AI blindly, they might think they found a "twin" hurricane, but it could just be a mathematical coincidence. They need a way to peek behind the curtain and check the actual weather photos.

2. The Solution: A "Provenance-Aware" Workbench

The authors built a tool that acts like a high-tech detective's dashboard. It connects the mathematical fingerprints directly back to the original weather photos and data.

• The "Experiment" Concept: Think of the tool as a laboratory bench. You can run different "experiments" side-by-side. One experiment might use AI Model A to create fingerprints; another might use Model B.
• The Link: The tool keeps a strict chain of custody. If you find a match in the math, you can click a button and instantly see the original satellite image, the exact time, and the location. It answers the question: "Did this match happen because the weather was similar, or just because the computer did something weird?"

3. How It Works in Practice (The Hurricane Example)

The paper demonstrates this tool using Tropical Cyclones (hurricanes) from the North Atlantic.

• Step 1: The Map: The tool creates a visual map of all the weather data. It groups similar weather patterns together.
• Step 2: The Check: The scientists see a cluster of points on the map. They click on it, and a gallery of actual hurricane photos pops up. They confirm, "Yes, this cluster really does contain hurricanes, not just random noise."
• Step 3: The Search: A scientist picks a specific patch of a hurricane (like the eye of Hurricane Matthew) and asks the computer: "Find me other times this exact patch of sky looked like this, but only in the Caribbean."
• Step 4: The Result: The system instantly finds matches, like Hurricane Irma and Hurricane Maria, showing the scientist the original photos and proving the match is real.

4. The "Magic" of Speed (Scalability)

Usually, searching through millions of these mathematical fingerprints requires a supercomputer with massive memory.

• The Innovation: The authors built a backend that acts like a smart librarian. Instead of dumping the entire library onto the desk (which would crash the computer), the librarian only pulls out the specific books needed for the search.
• The Result: They showed that this tool can search through 23 million weather fingerprints on a standard, off-the-shelf workstation computer without slowing down. It's fast enough to let a scientist ask a question, wait a split second, and get an answer.

Summary

This paper isn't about inventing a new weather model or predicting the future. It's about building a trustworthy search engine for the massive amounts of weather data we already have.

It gives scientists a way to:

1. Explore data using AI fingerprints.
2. Verify that those fingerprints actually make sense physically.
3. Search through millions of records instantly to find rare or extreme weather events that look like the one they are studying.

It turns a chaotic mountain of data into a navigable library where you can find the "twin" of any weather event, provided you have the right map to find it.

Technical Summary

1. Problem Statement

Earth system science is generating petabyte-scale, high-dimensional datasets from both physics-based models and emerging AI-based weather/climate models. While embedding-based representations (mapping complex data into vector spaces) offer a promising method for similarity search and analog retrieval, they introduce significant scientific challenges:

• Scientific Meaningfulness: Nearest neighbors in latent space may reflect genuine physical weather structures, or they may be artifacts of preprocessing, geography, seasonality, model bias, or the specific embedding architecture.
• Fragmentation: Current workflows are disjointed. Embedding generation, dimensionality reduction, vector search, model auditing, and meteorological visualization are typically handled by separate tools. This makes it difficult for researchers to trace latent-space results back to physical evidence or to compare different representation models effectively.
• Scalability: Embedding workflows can generate millions of vectors (e.g., per timestep or ensemble member), often exceeding the RAM capacity of standard workstations, necessitating out-of-core search solutions.

2. Methodology

The authors propose an open-source visual analytics workbench designed to bridge the gap between latent-space search and physical meteorological evidence. The system is built on three core design goals:

A. System Architecture

• Embedding Experiment Abstraction: The system treats each embedding set as an "experiment" linked to a shared source reference (source imagery, metadata, timestamps, spatial coordinates). This allows multiple models (e.g., Vision Transformers, Autoencoders) to be evaluated over the same source data without duplicating the underlying data.
• Provenance Tracking: The system maintains strict links between embedding vectors and their physical origins (source data, model configuration, preprocessing steps), ensuring that any search result can be traced back to the raw meteorological data.
• Separation of Concerns: Source data tables are separated from experiment tables (storing vectors and indexes), enabling flexible comparison of different representations.

B. Interactive Workflow

The workbench supports a two-stage discovery loop:

1. Latent-Space Inspection: Users visualize embedding spaces using dimensionality reduction (PCA, UMAP) linked to metadata brushing, parallel coordinates, and image galleries. This allows users to verify if latent clusters correspond to physical phenomena (e.g., storm tracks) rather than artifacts.
2. Retrieval Strategy Design: Users can construct queries at various scales:
   • Global: Entire atmospheric states.
   • Localized: Specific spatial patches.
   • Constrained: Filtered by metadata (e.g., storm intensity) or geography.
   • Staged: Global-to-local refinement.
   • Results are displayed with linked source imagery, spatial context, and similarity scores.

C. Scalable Backend

• Out-of-Core Search: To handle datasets exceeding workstation memory, the system uses Lance (a columnar data format) with disk-backed vector indexes.
• Indexing Strategy: It employs IVF-PQ (Inverted File Index with Product Quantization). IVF partitions the vector space to limit search scope, while PQ compresses vectors to reduce storage and memory footprints.

3. Key Contributions

1. Provenance-Aware Scientific Retrieval: A system architecture that connects embedding vectors directly to source imagery, spatial coordinates, and model configurations, ensuring traceability from latent space to physical data.
2. Interactive Retrieval-Strategy Design: A visual interface allowing domain scientists to design, test, and refine retrieval strategies (patch-level, metadata-constrained, etc.) and immediately verify if results reflect meaningful atmospheric structures.
3. Scalable Tropical Cyclone Case Study: A demonstration using ERA5 reanalysis data and IBTrACS storm metadata. The system successfully performed retrieval using DINOv3 (a vision foundation model) embeddings over tens of millions of high-dimensional vectors on commodity hardware.

4. Results

The paper presents a case study on Tropical Cyclone Retrieval and a scalability benchmark:

• Case Study (Tropical Cyclones):

  • Setup: ERA5 data (2016–2018) encoded as 3-channel RGB images (Pressure Anomaly, Wind Speed, Water Vapor) processed by DINOv3.
  • Inspection: UMAP visualization showed storm-containing composites forming a distinct peripheral cluster, separable from non-storm states.
  • Retrieval: A localized query on Hurricane Matthew (2016) restricted to the Caribbean successfully retrieved:
    1. Matthew's own track frames.
    2. Hurricane Irma (2017) and Hurricane Maria (2017) in the same region.
    3. Other 2017/2018 Atlantic systems crossing the filter region.
  • Outcome: The system successfully identified meteorologically similar events, validating that the latent space captured physical storm structures.

• Scalability Evaluation:

  • Hardware: Simulated a commodity workstation with 16 GiB RAM and local NVMe SSD.
  • Dataset: Scaled from 0.98M to 23.55M vectors (768-dimensional).
  • Memory Efficiency: Despite the raw float32 vector footprint exceeding 16 GiB at the largest scale, the peak Resident Set Size (RSS) remained around 3 GiB, proving effective out-of-core operation.
  • Performance:
    • Latency: Mean latency remained under 100 ms even for the largest dataset.
    • Accuracy: Recall@10 (agreement with exact nearest neighbors) stayed above 92%.
  • Conclusion: The system supports interactive exploration of massive embedding collections without requiring HPC-grade memory.

5. Significance

This work addresses a critical bottleneck in Earth system science: the transition from producing massive datasets to searching and interpreting them.

• Trustworthiness: By linking latent-space results to physical evidence, the workbench mitigates the risk of "hallucinated" or artifact-driven scientific conclusions.
• Flexibility: The "experiment" abstraction allows scientists to rapidly compare different AI models (e.g., foundation models vs. autoencoders) without rebuilding data pipelines.
• Accessibility: The demonstration of out-of-core search on commodity hardware democratizes access to large-scale embedding analysis, removing the need for specialized high-memory clusters.
• Future Pathway: The system serves as infrastructure for future work in AI-driven weather emulation, rare event discovery, and multi-modal (language/vision) interfaces for climate data.

In summary, the paper presents a robust, scalable, and scientifically grounded framework that transforms embedding-based search from a "black box" algorithm into an interactive, verifiable tool for meteorological discovery.