Seeing Above and Below the Canopy: Modeling and Interpreting Species Occupancy with Multimodal Habitat Representations

This paper introduces an interpretable, multimodal modeling framework that combines AI-derived satellite and ground-level imagery to create microhabitat-aware species occupancy models, significantly improving prediction accuracy and translating complex AI features into transparent, text-based ecological insights for conservation management.

The Gist

Imagine you are trying to figure out where a specific type of squirrel lives in a massive forest.

The Old Way: The "Blind Map"
Traditionally, ecologists (scientists who study nature) have used a "Blind Map" approach. They look at big, coarse data like "average temperature" or "soil type" for a whole region. It's like trying to guess where a squirrel lives just by knowing the average temperature of the entire state. You might get a general idea, but you miss the tiny details: Is there a hollow log nearby? Is the ground covered in pine needles? Is there a stream running through the bushes? These small details are the squirrel's actual home, but the "Blind Map" can't see them.

The New Way: The "Super-Eye"
This paper introduces a new method that gives ecologists "Super-Eyes." They combine two types of cameras:

1. Satellite Cameras: Looking down from space to see the big picture (like the shape of the forest).
2. Ground Cameras: Looking at the forest floor (like a squirrel's eye view) to see the tiny details (like moss on a rock or a pile of leaves).

Instead of just taking a photo and hoping a human can spot the squirrel, the researchers use AI (Artificial Intelligence) to "read" these photos. The AI doesn't just look for the squirrel; it learns to understand the vibe of the habitat. It learns that "this specific mix of shadows, leaves, and dirt" is a perfect squirrel home, even if the squirrel isn't in the picture at that exact moment.

The "Black Box" Problem
Here's the catch: The AI is a "Black Box." It's incredibly smart and can predict where squirrels live better than any human, but it's hard to ask it why. It's like a genius chef who makes a perfect stew but refuses to tell you the recipe. Ecologists need to know the recipe (e.g., "Oh, it's because of the oak trees!") to make conservation plans.

The Solution: The "AI Translator"
The authors created a clever translator to fix this.

1. The Detective Work: They ask the AI, "Which photos look most like a squirrel home?" and "Which look least like one?"
2. The Comparison: They use another AI (a language expert) to look at the "good" photos and the "bad" photos and describe the differences in plain English.
   • AI says: "The good photos have 'mossy logs' and 'dense ferns,' while the bad photos have 'bare dirt' and 'open sky.'"
3. The Translation: They turn these English descriptions ("mossy logs") back into simple numbers that the ecologists can use.

The Result
By doing this, they get the best of both worlds:

• High Performance: The model predicts animal locations much more accurately than before because it sees the tiny details the old maps missed.
• Clear Explanation: They can now tell a park ranger, "Don't just protect the whole forest; specifically protect the areas with mossy logs and dense ferns," because the AI told them exactly what matters.

In a Nutshell
This paper is about teaching computers to be better nature detectives. They use AI to see the tiny details of an animal's home that satellites miss, and then they use a "translator" to explain those findings in simple language so humans can actually use them to save species. It's like upgrading from a blurry, low-resolution map to a high-definition, annotated guidebook for nature.

Technical Summary

1. Problem Statement

Effective conservation requires accurate models of species-habitat relationships. Traditional Species Occupancy Models (SOMs) rely on coarse-scale environmental covariates (e.g., average temperature, elevation, soil type) which often fail to capture critical microhabitat features (e.g., understory structure, leaf litter, deadwood) that drive species distribution.

• Limitation of Current Methods: While recent deep learning approaches use high-resolution satellite imagery to improve predictions, they still miss ground-level details obscured by tree canopies. Conversely, ground-level camera trap images capture these microhabitats but are rarely integrated into statistical occupancy models in a scalable way.
• The Interpretability Gap: Deep learning models using image embeddings as predictors are often "black boxes." Ecologists need interpretable models to understand why a species occupies a site to inform management decisions, but high-dimensional image features lack direct semantic meaning.

2. Methodology

The authors propose a Deep Multimodal Occupancy Modeling framework that integrates three data modalities:

1. Environmental Covariates ($X_{env}$): Standard coarse-scale variables (WorldClim climate data, SoilGrids).
2. Satellite Imagery Embeddings ($X_{sat}$): High-resolution embeddings derived from overhead satellite images using the AlphaEarth Foundations model.
3. Ground-Level Imagery Embeddings ($X_{img}$): Embeddings derived from "blank" (no-animal) daytime camera trap images using the DINOv2 vision foundation model. These capture microhabitat structure invisible from space.

Core Technical Components:

• Bayesian Hierarchical Occupancy Model: The authors implement a logit-linear occupancy model (following MacKenzie et al.) that explicitly disentangles occupancy probability ($\psi$) from detection probability ($p$). This accounts for imperfect detection, a critical factor in ecological surveys.
• Biolith Framework: A new open-source Python package built on NumPyro (probabilistic programming) and JAX to efficiently fit these complex hierarchical models using Hamiltonian Monte Carlo (NUTS sampler) on GPUs.
• Multimodal Integration: The model concatenates $X_{env}$, $X_{sat}$, and $X_{img}$ to predict occupancy. Regularization (L1/L2 priors) is used to prevent overfitting given the high dimensionality of image embeddings.

Interpretability Pipeline (The "Explainability" Method):

To overcome the black-box nature of deep features, the authors introduce a three-phase method to generate human-readable habitat descriptions:

1. Ranking: Images are ranked by their logistic contribution to occupancy ($\beta_{img}^\top x_{img}$) to identify "likely occupied" vs. "likely unoccupied" sites.
2. Natural Language Summarization: Using an adaptation of VisDiff, the system compares captions of likely-occupied vs. unoccupied images (generated by BLIP-2) to extract natural language hypotheses about distinguishing habitat elements.
3. Quantification: The top 5 habitat descriptions are converted into continuous covariates ($X_{hbt}$) using CLIP (Contrastive Language-Image Pre-training) similarity scores. These text-based scores replace the raw image embeddings in the model, maintaining predictive power while providing interpretability.

3. Key Contributions

1. First Scalable Multimodal Occupancy Model: Demonstrates a mechanism to combine environmental variables, satellite imagery, and ground-level camera trap imagery within a rigorous Bayesian occupancy framework.
2. Microhabitat Awareness: Proves that ground-level imagery captures unique, fine-grained habitat information (e.g., understory density, deadwood) that satellite imagery and coarse variables miss.
3. Interpretability without Performance Loss: Introduces a novel method to distill high-dimensional image features into natural language habitat elements and re-quantify them as covariates. This allows ecologists to interpret model coefficients in terms of specific habitat features (e.g., "moss-covered logs") without sacrificing predictive accuracy.
4. Open-Source Tooling: Release of Biolith, a Python library bridging deep learning and statistical ecology, facilitating the adoption of these methods.

4. Results

The study was evaluated on 16 species across the contiguous United States using data from Wildlife Insights (training) and Snapshot USA (testing), with a strict 10km buffer to prevent spatial autocorrelation.

• Predictive Performance:

  • Deep features outperform baselines: Models using only traditional environmental covariates ($X_{env}$) performed no better than a null model (constant occupancy) for many species.
  • Multimodal gains: Adding satellite ($X_{sat}$) and/or ground-level ($X_{img}$) imagery significantly improved performance (measured by normalized Log Point-wise Predictive Density, LPPD$_{norm}$).
  • Complementarity: The combination of all three modalities yielded the best results. Ground-level imagery provided the largest average gain (+0.17 LPPD$_{norm}$), followed by satellite imagery (+0.10).
  • Species-Specific Insights:
    • Small mammals (e.g., Eastern Chipmunk, Gray Squirrel): Showed massive improvements with ground-level imagery, indicating strong dependence on microhabitat structure (understory, burrows) invisible from space.
    • Generalists/Carnivores: Showed mixed results; some (e.g., Raccoon) benefited from all modalities, while others (e.g., Black Bear) were better predicted by broad landscape features (satellite) alone.
    • Ungulates: White-tailed deer benefited from microhabitat context; Mule deer showed poor performance due to distribution shifts between train/test sets.

• Interpretability Validation:

  • Replacing raw image embeddings with the 5 text-derived CLIP scores ($X_{hbt}$) resulted in a negligible drop in performance (average drop of 0.05 LPPD$_{norm}$), proving that the natural language summaries effectively captured the predictive signal.
  • The extracted habitat elements (e.g., "rocky forest understory" for squirrels, "dirt roads" for bobcats) aligned with expert ecological knowledge.

5. Significance

• Bridging AI and Ecology: This work moves beyond simple "prediction" to "understanding." It provides a pathway to use powerful, data-driven AI models while retaining the interpretability required for scientific insight and policy-making.
• Actionable Conservation: By identifying specific microhabitat features (e.g., the need for deadwood or specific understory density) that drive occupancy, managers can design more effective restoration and habitat management strategies.
• Scalability: The method leverages existing, massive datasets (Wildlife Insights) and pre-trained foundation models, making it scalable to thousands of species and global regions without requiring manual labeling of habitat features.
• Future Directions: The paper highlights the potential for citizen science to collect targeted "useful" habitat data (not just species sightings) and suggests future work in disentangling occupancy vs. detection drivers within ground-level imagery.

In summary, the paper presents a robust framework for microhabitat-aware species modeling that combines the predictive power of multimodal deep learning with the interpretability of natural language, offering a new standard for ecological modeling and conservation planning.