Olfactory pursuit: catching a moving odor source in complex flows

1. Problem Statement

The paper addresses the challenge of olfactory pursuit: locating and intercepting a moving target (prey/source) using sparse, intermittent, and delayed sensory signals (odor) in turbulent-like environments.

• The Core Difficulty: Unlike stationary source localization, a moving target leaves a "chemical ghost"—an odor plume that lags behind the true target position due to turbulent diffusion.
• The Blind Spot: When the target exhibits persistent motion (moving in a straight line for extended periods) at speeds comparable to the searcher, purely exploratory strategies fail. The searcher cannot distinguish between the lingering odor trail and the target's future trajectory.
• Goal: Develop a decision-making framework that allows an agent to maintain a joint belief over the target's position and velocity to predict future states and intercept the target efficiently.

2. Methodology

The authors formulate the problem as a Partially Observable Markov Decision Process (POMDP) where the agent maintains a belief state $b_t(x, u)$ representing the probability distribution over the target's position ($x$) and velocity ($u$).

A. Models

Two distinct search scenarios were simulated:

1. Discrete Run-and-Tumble (R&T): Both agent and target move on a discrete lattice. The target changes direction with a persistence time $\tau_p$. Odor detection is modeled via a quasi-static likelihood derived from the diffusion equation.
2. Continuous Run-and-Tumble: The target moves in continuous time and space (off-lattice) with Brownian-like odor tracers that have finite lifetimes. The agent moves on a discrete lattice but uses an approximate model of the target dynamics and odor dispersion (introducing model mismatch).

B. Strategies Compared

1. Quasi-Optimal Policy (Benchmark):
   • Solved numerically using the SARSOP algorithm to approximate the Bellman equation for the POMDP.
   • This provides the theoretical lower bound for search time but is computationally expensive and infeasible for high-dimensional or continuous spaces.
2. Infotaxis (Exploration-Only):
   • A well-known strategy where the agent moves to minimize the expected entropy (uncertainty) of its belief map.
   • Effective for stationary targets but prone to failure when the target moves persistently, as it gets "stuck" following the odor trail rather than predicting the target's path.
3. Hybrid Heuristic Policy (Proposed):
   • A composite policy that linearly blends Information Gain (Infotaxis) and Exploitation (Greedy planning).
   • The Formula:
     $$A(t) = \arg\max_a \left[ w Q_{\text{greedy}}(a) - (1-w) H[b|a] \right]$$
     • $H[b|a]$: Expected entropy reduction (exploration).
     • $Q_{\text{greedy}}(a)$: A value function derived from a fully observable MDP (assuming the agent knows the target's exact state). This term penalizes actions that do not lead to immediate interception.
     • $w$: A mixing weight ($0 \le w \le 1$) tuned to balance exploration and exploitation.

3. Key Contributions

• Identification of the "Blind Spot": The paper rigorously demonstrates that purely exploratory strategies (Infotaxis) fail catastrophically when the target's persistence time is long, effectively reducing the search to a random walk.
• Predictive Inference as Key: The study establishes that successful pursuit in complex flows requires predictive inference of target motion (velocity and trajectory) rather than just uncertainty reduction.
• Robust Hybrid Policy: The authors propose a computationally efficient heuristic that dynamically balances information gathering with trajectory prediction. This policy adapts to the target's behavior:
  • Low Persistence: Relies more on Infotaxis (exploration) as the target is unpredictable.
  • High Persistence: Shifts to Greedy planning (exploitation) to intercept the predictable trajectory.
• Generalizability: The hybrid strategy remains effective even in the Continuous R&T model, where the agent operates with an imperfect internal model of the target dynamics and the odor plume is unsteady (lagging behind the source).

4. Results

• Discrete Model Performance:
  • The hybrid policy achieves near-optimal performance, closely matching the quasi-optimal POMDP solution across all persistence times ($\tau_p$).
  • Improvement over Infotaxis: For large $\tau_p$, the hybrid policy reduces average search time by approximately 70% compared to pure Infotaxis.
  • Optimal Weight: An optimal mixing weight $w \approx 0.9$ was found for high persistence, indicating a strategy that is mostly greedy with a small exploratory component.
• Continuous Model Performance:
  • Despite model mismatch (agent using a discrete approximation for continuous dynamics) and unsteady odor wakes, the hybrid policy outperformed Infotaxis by ~25% at intermediate persistence times.
  • The strategy showed robustness to variations in the ratio of target speed to odor diffusion speed ($U\tau_d/\lambda$).
• Behavioral Dynamics:
  • Visualizations (Supplementary Movies) show that the agent transitions from "spiraling" (exploratory) behavior at low $\tau_p$ to "direct chasing" (interceptive) behavior at high $\tau_p$.

5. Significance

• Biological Relevance: The findings explain how biological predators (e.g., sharks, wolves, plankton) might successfully hunt mobile prey in turbulent environments. It suggests that cognition involving motion prediction is a critical evolutionary adaptation, not just sensory sensitivity.
• Robotics and Engineering: The proposed hybrid heuristic offers a practical, computationally lightweight framework for autonomous systems (drones, underwater vehicles) tasked with tracking leaking chemical sources or moving targets in dynamic, information-poor environments.
• Theoretical Advancement: The paper bridges the gap between information-theoretic search (reducing entropy) and control-theoretic pursuit (minimizing capture time), demonstrating that the optimal solution lies in a dynamic balance between the two, particularly when the searcher and target move at comparable speeds.

Conclusion

The paper concludes that effective olfactory pursuit in complex flows is not merely about finding the source but anticipating its motion. By integrating a "greedy" value function derived from a fully observable control problem with an information-gain drive, agents can overcome the limitations of purely exploratory strategies and robustly intercept moving targets even in the presence of significant environmental noise and model uncertainty.