PATHFINDER: Multi-objective discovery in structural and spectral spaces

The paper introduces PATHFINDER, an autonomous microscopy framework that integrates novelty-driven exploration with multi-objective optimization to efficiently discover diverse and scientifically significant structural and spectral states while avoiding premature convergence on single solutions.

The Gist

Imagine you are a detective trying to solve a mystery in a massive, foggy city. You have a limited amount of time and fuel (your "experimental budget"), and you need to find the most important clues to understand how the city works.

In the past, automated scientists (robots doing experiments) were like detectives with a very narrow focus. They were told, "Find the biggest, shiniest building." So, the robot would drive straight to the tallest skyscraper, measure it a thousand times, and stop. It missed the weird, hidden alleyways, the strange old houses, or the quiet parks that might hold the real secrets of the city. It got stuck on the "obvious" answer and stopped looking.

PATHFINDER is a new, smarter detective framework designed to fix this. It doesn't just look for the "best" thing; it looks for the most interesting and most useful things, balancing two goals at once.

Here is how PATHFINDER works, broken down into simple concepts:

1. The Two Maps (Structural vs. Spectral)

Imagine the city has two different maps:

• Map A (The Look): This shows what things look like from the outside (the structure). Is it a brick house? A glass tower? A muddy field?
• Map B (The Function): This shows what things do or how they react (the spectrum). Does the house hum with electricity? Does the glass glow under UV light?

Old robots usually picked a spot based on Map A, then measured it, or picked a spot based on Map B, then measured it. They rarely used both maps together to make a smart decision.

2. The "Novelty" Radar (Finding the Weird Stuff)

PATHFINDER has a special radar for Novelty.

• The Old Way: If you see 1,000 red houses, the robot ignores them because they are boring.
• The PATHFINDER Way: It asks, "Have I seen a blue house yet? Or a house made of jelly?" If the answer is no, it marks that spot as "High Priority," even if the blue house doesn't look very important at first glance. It wants to explore the weird, rare, and unknown parts of the city so it doesn't miss a new discovery.

3. The "Reward" Compass (Finding the Useful Stuff)

At the same time, PATHFINDER has a compass for Reward.

• This asks, "Where is the action?" If a specific type of house is known to produce a lot of energy, the robot wants to measure those spots to understand why.
• It builds a "guessing game" (a mathematical model) to predict where the best energy spots are, based on the few measurements it has already taken.

4. The Balancing Act (The "Pareto" Dance)

This is the magic sauce. PATHFINDER doesn't choose between "Weird" and "Useful." It tries to find the perfect balance.

Imagine you are packing a backpack for a hiking trip.

• If you only pack Useful items (water, food), you might get lost because you didn't bring a map of the weird trails.
• If you only pack Novel items (a rock from a volcano, a strange flower), you might get hungry because you didn't bring food.
• PATHFINDER packs the perfect mix: a few useful items and a few items from places you've never been before. It ensures that every step you take teaches you something new about the landscape and helps you solve the main problem.

How It Works in Real Life

The paper tested this on two types of "cities":

1. A Digital City (STEM-EELS): They used a massive database of images and spectra of nanoparticles. PATHFINDER was able to find rare particles that other methods missed, proving it could navigate a complex map without getting stuck.
2. A Real-Time City (Ferroelectric Microscopy): They used a real microscope to scan a material called Lead Titanate. As the robot scanned, it had to decide in the moment where to look next.
   • It started by taking a few random samples (seeds).
   • Then, it looked at the "Look" map to find weird textures.
   • It looked at the "Function" map to find high-energy spots.
   • It combined them to pick the next spot.
   • Result: Instead of circling the same high-energy spot over and over, it jumped to new, strange textures that turned out to have surprising electrical properties.

The "Human-in-the-Loop" Safety Net

Sometimes, the robot might get confused or the data might be noisy. PATHFINDER allows a human scientist to step in. If the robot is about to drive off a cliff (make a bad guess), the human can say, "Wait, look over there instead," or "Change the rules." This makes the system safe and adaptable, like a co-pilot.

The Bottom Line

PATHFINDER changes the goal of science from "Find the best answer as fast as possible" to "Explore the whole map to find the most interesting and useful answers."

It prevents scientists from getting stuck in a loop of doing the same thing over and over. Instead, it encourages the robot to be curious, to visit the "weird" neighborhoods of the data, and to build a complete picture of how materials work, ensuring that no rare, groundbreaking discovery is left behind just because it didn't look like the obvious winner.

Technical Summary

1. Problem Statement

Autonomous microscopy and materials characterization often rely on machine learning (ML) workflows that optimize a single predefined objective (e.g., maximizing a specific spectral signal or finding a specific defect). This approach suffers from several critical limitations:

• Premature Convergence: Algorithms tend to exploit locally promising regions, collapsing onto a single "apparent optimum" and overlooking rare but scientifically significant states.
• Limited Scope: Most workflows fail to coordinate exploration across structural (image-based), spectral (functional response), and measurement spaces simultaneously.
• Rigid Policies: Fixed policies or single-objective rewards cannot adapt to evolving scientific hypotheses or handle the trade-offs between exploring new structural motifs and exploiting known high-value functional responses.
• Human-ML Disconnect: Existing "human-in-the-loop" frameworks often intervene only when the ML fails, rather than integrating human guidance into the fundamental decision-making logic for balancing novelty and optimization.

The core challenge is to design an autonomous system that balances target-driven optimization (finding the best functional response) with novelty discovery (finding structurally unique states) under finite experimental budgets (time, dose, probe damage).

2. Methodology: The PATHFINDER Framework

The authors introduce PATHFINDER (Policy for Active, Time-aware, Human-in-the-loop, Fidelity-adaptive Imaging, Navigation, and Discovery with Efficient Rewards), a multi-objective autonomous microscopy framework.

Core Architecture

The framework operates via two coupled branches managed by a multi-objective Bayesian optimization controller:

1. Structural Reward ($R_1$): Novelty in Feature Space
   • Mechanism: Uses a Variational Autoencoder (VAE) to encode local image patches (structural data) into a low-dimensional latent space.
   • Metric: $R_1$ is defined as the structural novelty, calculated as the distance of a candidate patch from previously measured points in the latent space.
   • Formulations:
     • Static: Distance from the global latent mean (rarity relative to the whole dataset).
     • Adaptive: Distance from the set of already measured points (rarity relative to the current trajectory), which actively prevents the agent from revisiting similar regions.
2. Functional Reward ($R_2$): Optimization in Spectral Space
   • Mechanism: Uses a Surrogate Model (Deep Kernel Learning with Gaussian Processes) to predict the functional response (e.g., EELS intensity, ferroelectric nonlinearity) based on structural inputs.
   • Metric: $R_2$ represents the target physical property (e.g., signal intensity, hysteresis loop area).
3. Decision Making: Multi-Objective Bayesian Optimization
   • Instead of maximizing a single scalar reward, PATHFINDER maximizes the Expected Hypervolume Improvement (EHVI).
   • This acquisition function selects the next measurement location that expands the Pareto front between structural novelty ($R_1$) and functional performance ($R_2$).
   • This ensures the system does not collapse to a single corner of the objective space but explores diverse trade-offs.

Workflow Steps

1. Initialization: A small "seed" set of measurements is taken.
2. Encoding: All candidate locations are encoded into the VAE latent space to compute $R_1$.
3. Surrogate Update: The surrogate model for $R_2$ is updated with new experimental data.
4. Acquisition: The algorithm calculates the EHVI for all unmeasured candidates and selects the location with the highest score.
5. Iteration: The experiment proceeds sequentially, updating the Pareto front and the novelty landscape at each step.

3. Key Contributions

• Dual-Objective Discovery: The first framework to explicitly couple latent-space structural novelty with spectral functional optimization in a single autonomous loop.
• Adaptive Novelty Definition: Introduces an "adaptive" novelty metric where the definition of "new" evolves based on the measurement history, preventing the agent from getting stuck in local structural basins.
• Pareto-Based Exploration: Moves beyond single-objective optimization to a multi-objective approach that preserves diversity in the structure-property landscape.
• Human-in-the-Loop Integration: The framework is designed to accept human priors and priorities, allowing the scientific intent to guide the balance between exploration and exploitation dynamically.

4. Experimental Results

The framework was validated using two distinct systems:

A. Pre-acquired Benchmark: STEM-EELS Nanoparticles

• System: Fluorine- and tin-co-doped indium oxide nanoparticles.
• Setup: Used a pre-acquired dataset where both High-Angle Annular Dark-Field (HAADF) images and Electron Energy Loss Spectroscopy (EELS) spectra were available for the entire field of view.
• Outcome:
  • The algorithm successfully identified a broad set of Pareto-optimal trade-offs between structural novelty and spectral signal.
  • Unlike single-objective optimizers that collapsed to a single high-signal region, PATHFINDER maintained structural diversity, sampling distinct regions in the VAE latent space.
  • It effectively located rare structural motifs that possessed significant spectral responses, which would have been missed by standard greedy acquisition.

B. On-the-Fly Experiment: Ferroelectric Scanning Probe Microscopy (SPM)

• System: Epitaxial $PbTiO_3$ thin films on $KTaO_3$ substrates.
• Setup: A live experiment starting from a single structural scan. The agent sequentially acquired local force spectroscopy (nonlinearity coefficient) at selected points.
• Outcome:
  • Adaptive vs. Static: The "adaptive" formulation (novelty relative to measured history) achieved significantly higher coverage of the latent manifold than the "static" formulation.
  • Trajectory: The acquisition trajectory was non-uniform, targeting domain boundaries and interfaces (high structural novelty) while also refining regions of high nonlinearity.
  • Decoupling: The results confirmed that structurally unusual regions are not always the same as those with the highest functional response, validating the need for a multi-objective controller.
  • Dynamic Behavior: The functional reward ($R_2$) showed non-monotonic evolution (local refinement followed by global redirection), characteristic of an uncertainty-aware search rather than simple exploitation.

5. Significance and Outlook

• Paradigm Shift: PATHFINDER shifts autonomous microscopy from a purely optimization-driven mode (finding the "best" point) to a discovery-oriented mode (mapping the "structure-property landscape").
• Efficiency: It maximizes the scientific return on finite experimental budgets by ensuring that every measurement contributes to either expanding structural knowledge or refining functional understanding, avoiding redundant sampling.
• Scalability: The framework is generalizable to other multimodal techniques (e.g., combining optical, electron, and scanning probe microscopy) and can integrate external priors or human guidance seamlessly.
• Future Impact: This approach paves the way for "self-driving laboratories" that can autonomously formulate and test complex hypotheses, discovering rare materials states and novel structure-property relationships that fixed-policy pipelines would inevitably miss.