ForestPersons: A Large-Scale Dataset for Under-Canopy Missing Person Detection

This paper introduces ForestPersons, a large-scale dataset of 96,482 under-canopy images with over 200,000 annotations designed to address the limitations of aerial UAV imagery in detecting missing persons during forest Search and Rescue missions.

The Gist

Imagine you are looking for a friend who has gone missing in a massive, dense forest. You have a drone, but here's the problem: the trees are the boss.

If you fly your drone high up in the sky (like a bird), you get a great view of the whole forest, but the thick leaves and branches act like a giant, green curtain. Your friend is hiding behind that curtain, and from high up, they look like a tiny speck, or they are completely invisible. This is how most current search-and-rescue drones work, and it's why they often fail to find people in deep woods.

Enter "ForestPersons": The Ground-Level Detective Kit.

This paper introduces a brand-new, massive dataset (a giant collection of training data) called ForestPersons. Think of it as a "school" for AI, but instead of teaching it to recognize cats and dogs in a living room, it teaches the AI how to find lost people under the trees.

Here is the breakdown of what makes this special, using some simple analogies:

1. The "Bird's Eye" vs. The "Hiker's Eye"

• Old Way (The Bird): Most search drones fly high up. It's like trying to find a specific person in a crowded stadium by looking down from a helicopter. You see the whole crowd, but you can't see the person hiding behind a row of seats.
• New Way (The Hiker): ForestPersons teaches the AI to look from ground level (or very low in the air). It's like sending a hiker into the woods. They can see through the gaps in the branches, spot a jacket on the ground, or see a hand waving from behind a bush. This dataset simulates exactly what a small, agile drone (called a MAV) would see when flying low through the trees.

2. The "Costume Party" of Poses

In normal city streets, people are usually standing up and walking. But in a forest search, a missing person might be:

• Lying down (exhausted or injured).
• Sitting (trying to rest).
• Hidden behind a tree trunk.

Most old AI training sets are like a photo album of people standing in a park. They don't know what to do when someone is lying in the dirt. ForestPersons is like a costume party where everyone is posing in weird, difficult ways. It forces the AI to learn that a "person" isn't just a standing stick figure; it's a shape that could be flat on the ground or half-hidden by a bush.

3. The "Seasonal Challenge"

The forest changes.

• Summer: Thick green leaves block the view.
• Winter: Bare branches and snow make the person stand out, but the snow can also hide them.
• Rain/Fog: The air gets murky.

This dataset includes photos from all these seasons and weather conditions. It's like training a soldier not just for a sunny day, but for rain, snow, and fog, so they don't get confused when the weather changes.

4. Why the Old AI Failed (The "Domain Gap")

The researchers tested old AI models (trained on city data or high-altitude drone data) on this new forest data.

• The Result: The old AI got a "F" grade. It was like giving a chef who only knows how to cook steak a menu of sushi. The tools didn't match the job.
• The Fix: When they trained new models specifically on ForestPersons, the AI suddenly got much smarter. It learned to ignore the leaves and focus on the human shape, even when only 20% of the person was visible.

5. The "Real-World" Test

The researchers even built a tiny test flight using a real drone to see if the AI trained on their photos would work in real life.

• The Surprise: Even though the photos were taken by hand-held cameras (not drones), the AI learned so well that it worked perfectly on the real drone footage. It's like practicing your driving in a parking lot and then being able to drive perfectly on a highway.

The Big Picture

ForestPersons is a public gift to the world. It's a massive library of "what it looks like to be lost in the woods." By making this data free, the researchers hope that:

1. AI gets better: Developers can build smarter drones that don't get fooled by leaves.
2. More lives are saved: If a drone can spot a person hiding under a tree 10 times better than before, that person gets found faster, and the rescue is more successful.

In short: They built a better "eye" for robots to look under the trees, so no one gets left behind in the woods.

Technical Summary

1. Problem Statement

Search and Rescue (SAR) missions in forest environments face a critical bottleneck: detecting missing persons under dense forest canopies.

• Limitations of Current Approaches: Existing UAV-based SAR datasets and models primarily rely on high-altitude, top-down, or oblique aerial perspectives. In dense forests, these views are severely obstructed by foliage, causing missing persons to be hidden or reduced to a few pixels, making detection unreliable.
• Gap in Existing Benchmarks: While ground-level datasets (e.g., COCO, CrowdHuman) exist, they typically feature upright, fully visible individuals in urban or open settings. They lack the specific challenges of SAR scenarios: heavy vegetation occlusion, non-standing poses (sitting/lying), and variable seasonal lighting.
• The Need: There is a lack of large-scale, annotated data specifically designed for under-canopy perspectives (ground-level or low-altitude), which are necessary for Micro Aerial Vehicles (MAVs) to effectively locate missing persons hidden beneath trees.

2. Methodology and Dataset Construction

The authors introduce ForestPersons, a novel large-scale dataset designed to bridge the gap between current benchmarks and real-world under-canopy SAR conditions.

Data Collection

• Perspective: Simulated MAV flight paths at low altitudes (1.5m – 2.0m) using handheld and tripod-mounted cameras to mimic the ground-level view of a drone navigating under a canopy.
• Scale: Contains 96,482 images and 204,078 annotated person instances.
• Diversity:
  • Environments: Dense forest interiors, valleys, and forest edges.
  • Conditions: Captured across different seasons (Summer, Fall, Winter with snow), weather (Clear, Overcast, Rain), and times of day (Afternoon, Dusk).
  • Subjects: Volunteers simulating missing persons in various states: Standing, Sitting, and Lying.
• Privacy: All faces were anonymized using automated detection and manual review.

Annotation Schema

Each instance includes:

1. Bounding Box: Delineating only the visible portion of the person (crucial for occlusion-aware training).
2. Pose: Categorized as Standing, Sitting, or Lying.
3. Visibility Level: A quantitative metric (20, 40, 70, 100) representing the degree of occlusion (from "almost unrecognizable" to "fully visible").

Dataset Splitting Strategy

To prevent data leakage and ensure fair evaluation, the dataset was split at the video sequence level rather than the frame level.

• Difficulty-Aware Splitting: Sequences were assigned a difficulty score based on the detection performance of a pre-trained Faster R-CNN ($1 - AP_{50}$).
• Distribution: Training, Validation, and Test sets were constructed to ensure balanced distributions of difficulty, pose, visibility, season, and weather across all splits.

Supplementary Data

• ForestPersonsIR: A companion thermal infrared dataset (64,142 images) is also released to support multi-modal SAR research.

3. Key Contributions

1. First Under-Canopy Benchmark: ForestPersons is the first large-scale dataset explicitly designed for missing person detection beneath dense forest canopies, addressing the "viewpoint gap" in existing SAR benchmarks.
2. Rich Attribute Annotation: Unlike standard datasets, it provides explicit labels for pose (standing/sitting/lying) and visibility/occlusion levels, enabling granular analysis of detection failures in complex environments.
3. Comprehensive Baselines: The paper evaluates a wide range of state-of-the-art object detectors (YOLO series, Faster R-CNN, Deformable R-CNN, DETR, DINO, CZ Det) and Vision-Language Models (VLMs) on this dataset.
4. Real-World Validation: The authors conducted experiments on a real-world MAV dataset to verify that models trained on their handheld-collected data generalize well to actual drone footage, despite domain gaps like motion blur.

4. Experimental Results

Performance of Standard Models

• Baseline Performance: Even top-performing models struggle compared to open-area detection. The best model, Deformable R-CNN, achieved an AP of 66.3 on the test set. CZ Det (designed for UAVs) and YOLOv11 also performed well (AP ~65.6).
• Failure of Prior Datasets: Models trained on existing UAV SAR datasets (e.g., SARD, WiSARD) or ground-level datasets (e.g., COCO, CityPersons) showed severe performance degradation (AP dropping to <12% or even 0.2%) when tested on ForestPersons. This confirms that prior benchmarks do not generalize to under-canopy scenarios.

Impact of Attributes

• Visibility: Detection performance correlates strongly with visibility. Models perform significantly better on visible (100%) instances compared to heavily occluded (20%) ones.
• Pose Generalization: Models trained only on standing poses failed to detect sitting or lying persons. Training on diverse poses is essential for SAR.
• Seasonal Generalization: Models trained only on summer data struggled in winter, and vice versa. Only models trained on all seasons achieved robust performance, highlighting the need for seasonal diversity.

Zero-Shot VLM Evaluation

• Generative MLLMs: State-of-the-art generative models (GPT-4o, GPT-5, Gemini 2.5) performed poorly (AP < 2.2) due to architectural mismatches in coordinate regression.
• Open-Vocabulary Detectors: Models like Grounding DINO and OWL-ViT performed better (AP ~52%) but still lagged significantly behind models fine-tuned on ForestPersons (AP ~64%), proving that domain-specific training is irreplaceable for this task.

Real-World MAV Validation

• Models trained on the original (handheld) ForestPersons dataset outperformed models trained on augmented data (simulated motion blur) and existing UAV datasets when tested on a real MAV flight dataset. This suggests that the visual features learned from the ground-level perspective are highly transferable to low-altitude drone operations.

5. Significance and Future Directions

• Advancing Autonomous SAR: ForestPersons provides the necessary data foundation to develop AI models capable of navigating and searching dense forests, a critical step toward fully autonomous SAR missions.
• Methodological Insights: The study demonstrates that "more data" is not enough; the data must match the specific viewpoint, occlusion patterns, and pose distributions of the target environment.
• Future Work: The authors propose using generative models (trained on ForestPersons) to synthesize extreme SAR scenarios (e.g., injured or buried victims) and integrating flight trajectory planning to optimize camera viewpoints for detection.

Conclusion: ForestPersons addresses a critical data scarcity in the field of computer vision for emergency response. By shifting the focus from high-altitude aerial views to under-canopy perspectives and providing rich annotations for occlusion and pose, it enables the development of robust detection systems that can significantly improve the success rates of real-world missing person rescues.