💻 computer science

A strongly annotated passive acoustic dataset for tropical bird monitoring

This paper introduces PteroSet, a strongly annotated passive acoustic dataset comprising over 73 hours of recordings and 15,000+ time-frequency labels for 168 Neotropical bird species in Colombia, designed to advance machine learning for biodiversity monitoring by addressing the scarcity of annotated tropical soundscapes and serving as a realistic benchmark for detection challenges.

Original authors: Daniela Ruiz, Juan Sebastián Ulloa, Zhongqi Miao, Nicolás Betancourt, Maria Paula Toro-Gómez, Andrés Hernández, Bruno Demuro, Eliana Barona-Cortés, Angela Mendoza-Henao, Andrés Sierra-Ricaurte, Sebast

Published 2026-05-21

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Daniela Ruiz, Juan Sebastián Ulloa, Zhongqi Miao, Nicolás Betancourt, Maria Paula Toro-Gómez, Andrés Hernández, Bruno Demuro, Eliana Barona-Cortés, Angela Mendoza-Henao, Andrés Sierra-Ricaurte, Sebastián Pérez-Peña, Rahul Dodhia, Pablo Arbeláez, Juan M. Lavista Ferres

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine trying to listen to a single conversation in a crowded, noisy stadium where thousands of people are talking, shouting, and singing all at once. Now, imagine doing that not for one hour, but for days, and trying to figure out exactly who said what, when they said it, and what species of bird they were. That is the challenge of listening to the tropical rainforest.

This paper introduces PteroSet, a massive new "library" of sound recordings and a detailed "map" of what's happening in those sounds, designed to help computers learn how to listen to birds in the tropics.

Here is a breakdown of the paper using simple analogies:

1. The Problem: The "Needle in a Haystack"

Scientists have been using microphones to record nature for a long time. It's like having a security camera that never sleeps. But these cameras record everything—wind, rain, insects, and cars.

The Issue: In the tropics, the "haystack" (the background noise) is huge, and the "needles" (specific bird calls) are hard to find.
The Gap: While we have great computer programs to recognize bird songs, they usually learn from recordings of birds in cooler, quieter places (like Europe or North America). When you take those programs to the noisy, complex tropics, they get confused. We needed a "training manual" specifically for the tropical jungle.

2. The Solution: PteroSet (The "Tropical Sound Library")

The researchers created a new dataset called PteroSet. Think of this as a very specific, high-quality training kit for AI.

Where: They recorded in two very different places in Colombia: the Amazon foothills (Putumayo) and the Caribbean lowlands (Magdalena). One area is a rapidly changing jungle, and the other is a more broken-up, farmed landscape.
What: They collected 73.6 hours of audio. But here's the trick: they didn't just dump the raw files. They created a "time-lapse" version. Imagine taking a 10-second clip every 30 minutes for a whole day and stitching them together into one 8-minute movie. This lets humans look at a whole day's worth of sound in a few minutes.
The "Map" (Annotations): This is the most important part. Humans (experts who know bird songs) listened to these clips and drew boxes around every bird call. They marked exactly when it started and stopped, and what pitch it was.
- They found 15,372 distinct bird events.
- They identified 168 different species.
- They labeled 6,702 of those calls down to the exact species name (like identifying a specific person in a crowd).

3. The Format: A New "Universal Translator"

Usually, sound data is messy. One researcher saves files in one way, another in a different way. It's like trying to build a puzzle where the pieces from different boxes don't fit together.

The Innovation: The team created a new file format (a JSON schema) based on a standard used for image recognition (called COCO).
The Analogy: Imagine they invented a universal "plug" that fits into any computer system. Now, instead of having to re-invent the wheel every time someone wants to study bird sounds, they can just plug this dataset in, and the computer knows exactly how to read the labels, the time, and the species.

4. The Test: Teaching the Computer to Listen

To prove this dataset works, the researchers taught a computer (using a model called ResNet-18) to answer a simple question: "Is there a bird in this 5-second clip, or not?"

The Challenge: They didn't just test it on the same data it learned from. They used a "leave-one-out" test. They trained the computer on data from four projects and then tested it on the fifth project it had never seen before. This is like studying for a test using four different textbooks and then taking a test based on a fifth one you've never opened.
The Result: The computer got about 72% to 75% accuracy.
- Why not 100%? Because the tropical jungle is hard! The paper notes that the computer struggled when the birds were quiet, when the sounds overlapped (two birds singing at once), or when the environment changed (like moving from the Amazon to the Caribbean).
- The Takeaway: The computer did well, but the dataset successfully showed where it struggles. This is valuable because it tells scientists exactly what problems they need to solve next.

5. Why This Matters (According to the Paper)

The paper claims this dataset is a "realistic benchmark."

Realism: Most other datasets are like a quiet library. PteroSet is like a busy market. It includes the messy reality of the tropics: birds singing over each other, background noise, and different habitats.
Open Access: The researchers are giving this library away for free. They want other scientists to use it to build better tools for protecting biodiversity.

Summary

In short, the authors built a giant, carefully labeled audio library of tropical birds in Colombia. They didn't just record the sounds; they drew a detailed map of every bird call. They packaged it in a new, easy-to-use format and proved that while computers are getting good at listening to nature, the tropical jungle is still a tough place for them to learn. This dataset is the new "textbook" that will help the next generation of AI become better at protecting the world's most diverse ecosystems.

Technical Summary: PteroSet – A Strongly Annotated Passive Acoustic Dataset for Tropical Bird Monitoring

Problem Statement

Passive acoustic monitoring (PAM) has become a critical, non-invasive tool for assessing biodiversity, particularly in remote and dense habitats where traditional surveys are impractical. While the scale of modern PAM datasets necessitates machine learning (ML) for automated sound detection and classification, the adoption of supervised deep learning methods is hindered by a scarcity of high-quality, time-resolved annotated datasets. This gap is especially acute in tropical regions, which harbor exceptional biodiversity but remain underrepresented in global data infrastructures. Existing datasets are often biased toward temperate regions, weakly annotated (clip-level labels only), or lack the detailed time-frequency bounding required for robust model training. Consequently, models trained on temperate data often underperform in complex tropical soundscapes characterized by dense acoustic overlap, high species turnover, and significant domain shifts across sites.

Methodology

Data Collection

The study collected audio data from two ecologically contrasting sites in Colombia: Puerto Asís (Putumayo Department), representing the Andean-Amazonian transition zone, and Pivijay (Magdalena Department), representing a historically transformed Caribbean lowland matrix.

Hardware: Autonomous recording units (AudioMoth) equipped with omnidirectional microphones.
Protocol: Devices recorded 1 minute every 30 minutes over a 24-hour period, yielding 48 minutes of audio per day at 192 kHz, 16-bit resolution.
Scope: Data was collected across five monitoring seasons between 2023 and 2025.

Data Annotation

To ensure high-quality training data, the authors developed a standardized protocol for "strong" annotations (time-frequency bounded labels).

Time-Lapse Generation: To manage the volume of data, the first 10 seconds of each of the 48 daily recordings were concatenated into a single 480-second "time-lapse" file, preserving 24-hour temporal coverage while reducing manual workload.
Expert Annotation: Four specialists used Raven Pro software to manually identify and delimit individual vocal events. Annotations included explicit onset ( $t_{min}$ ), offset ( $t_{max}$ ), lower frequency ( $f_{min}$ ), and upper frequency ( $f_{max}$ ) bounds.
Taxonomic Resolution: Species were identified based on diagnostic acoustic features using a standardized alphanumeric coding system (e.g., MEGCHO for Megascops choliba). Only signals clearly identified both acoustically and visually were annotated; faint or ambiguous vocalizations were excluded.
Filtering: The dataset was filtered to retain only biological sounds (BIO) and specifically avian vocalizations (AVEVOC), excluding non-biological noise and other taxa.

Computational Processing

Resampling: Audio was resampled from 192 kHz to 48 kHz for analysis, as bird vocalizations in this dataset rarely exceed 24 kHz.
Schema Unification: Annotations were converted from Raven Pro text files into a unified JSON schema inspired by the Common Objects in Context (COCO) image annotation format. This schema includes dataset metadata, category definitions, sound file information, and time-frequency annotations.
Baseline Model: A binary bird detection task was formulated. Audio was transformed into 5-second mel-spectrograms (2048-point FFT, 512-sample hop, 224 mel bands). A sliding window approach (1-second step) generated training samples.
Architecture: A ResNet-18 backbone, initialized with ImageNet weights and modified for single-channel input, was trained for binary classification (bird presence vs. absence).
Validation: A leave-one-project-out cross-validation strategy (5 folds) was employed to ensure strict temporal and spatial separation between training and testing data, rigorously testing model generalization across different recording sites.

Key Contributions

PteroSet Dataset: A curated collection of 563 time-lapse recordings (73.62 hours) containing 15,372 time-frequency annotations, including 6,702 species-level labels across 168 Neotropical bird species. This significantly expands the scale of highly annotated tropical audio data.
Realistic Tropical Benchmark: The dataset explicitly captures challenges inherent to Neotropical soundscapes, such as dense acoustic co-occurrence (multiple species singing simultaneously) and domain shifts between recording sites, which are often underrepresented in temperate datasets.
Annotation Schema: Introduction of a COCO-inspired JSON format adapted for bioacoustics, facilitating interoperability and integration with other datasets.
Reference Baseline: Provision of a reproducible ResNet-18 baseline for binary bird detection, establishing performance metrics for future comparisons.
Open-Access Resources: Full release of raw and processed data, annotation files, and code to support reproducibility.

Results

The baseline ResNet-18 model demonstrated effective discrimination between bird and non-bird acoustic events across diverse conditions, though performance varied by site.

Overall Performance: Across five cross-validation folds, the model achieved a mean F1-score of 0.718 ± 0.039, an AUPRC of 0.797 ± 0.047, and an accuracy of 0.850 ± 0.091.
Domain Shift: Performance varied significantly between folds. The model performed weakest on the MAP1 (Magdalena) test set (F1: 0.650) and strongest on the PPA4 (Putumayo) test set (F1: 0.742), highlighting the challenge of generalizing across ecologically distinct regions.
Acoustic Complexity: The dataset exhibits high acoustic density; approximately 19.8% of annotations involve temporal overlaps, and 60% of overlapping pairs also overlap in the frequency domain. This confirms that a significant portion of the data contains simultaneous vocalizations from multiple species.
Error Analysis: Qualitative analysis showed the model successfully identified clear, high-SNR vocalizations but struggled with faint calls and non-biological sounds that shared spectral characteristics with bird calls (e.g., insect tonal patterns).

Significance

The paper positions PteroSet not merely as a data repository but as a necessary step toward equitable and effective bioacoustic monitoring in the tropics. By addressing the mismatch between the biological richness of the Neotropics and the scarcity of digital representation, the dataset serves as a rigorous training resource and a challenging benchmark. It highlights that successful automated monitoring in these environments requires models capable of handling dense acoustic overlap, domain shifts, and long-tailed species distributions. The authors argue that centering these real-world complexities, rather than relying on cleaner, temperate datasets, is critical for developing models that generalize to global ecosystems. The adoption of a unified annotation schema further aims to lower barriers for building large-scale, multi-site training corpora and facilitating reproducible benchmarking in the bioacoustics community.