Detection and Classification of Cetacean Echolocation Clicks using Image-based Object Detection Methods applied to Advanced Wavelet-based Transformations

This thesis proposes and validates CLICK-SPOT, a deep learning framework that leverages wavelet-based transformations instead of traditional spectrograms to improve the automatic detection and classification of cetacean echolocation clicks in complex, noisy underwater environments.

The Gist

Imagine you are trying to listen to a conversation in a crowded, noisy room where everyone is whispering, shouting, and clapping. Now, imagine that conversation is happening underwater, where the "people" are killer whales, and the "clapping" is them using sonar to hunt and talk to each other.

This is the challenge Christopher Hauer tackled in his Master's thesis. He wanted to build a computer program that could automatically listen to hours of underwater recordings, find the specific sounds killer whales make (called "clicks"), and figure out which sounds are the whale talking and which are just the sound bouncing off a rock or the water surface (an "echo").

Here is the story of how he did it, broken down into simple concepts.

The Problem: The "Human Bottleneck"

Killer whales are social and use clicks constantly. Sometimes they click to hunt (like a sonar ping), and sometimes they click just to chat.

• The Issue: To understand what the whales are saying, scientists need to listen to thousands of hours of audio and manually mark every single click.
• The Reality: It takes a human expert about 12 hours to label just one minute of audio. If a whale clicks 150 times in a second, the human has to find, mark, and decide if it's a click or an echo for every single one. It's like trying to count grains of sand on a beach by picking them up one by one. It's impossible to do at scale.

The Solution: Teaching a Computer to "See" Sound

Christopher realized that computers are great at looking at pictures, but bad at listening to raw sound waves. So, he decided to turn the sound into pictures.

1. The Three Lenses (Waveform, Spectrogram, Scalogram)

Think of a sound recording as a piece of music.

• The Waveform: This is like looking at the music sheet. It shows the volume going up and down over time.
• The Spectrogram: This is like a heat map. It shows which notes (frequencies) are playing at which time. But, there's a catch: you can't see the exact time a note starts and the exact pitch it has at the same time perfectly (like trying to take a photo of a fast-moving car; if you focus on the speed, the car looks blurry).
• The Scalogram (The Magic Lens): Christopher used a special mathematical trick called a "Wavelet Transform." Imagine this as a smart camera lens that automatically zooms in for fast, high-pitched sounds (like a click) and zooms out for slow, low-pitched sounds. This gave the computer a much clearer picture of the tiny, split-second clicks.

He combined these three views into a single colorful image (Red = Waveform, Green = Scalogram, Blue = Spectrogram). To the computer, a whale click didn't look like a sound; it looked like a bright, sharp cone sticking out of a noisy background.

The Detective Work: YOLO and the "Box"

Christopher used a famous AI system called YOLO (You Only Look Once).

• The Analogy: Imagine YOLO is a security guard looking at a crowd. Its job is to draw a box around anyone who looks suspicious.
• The Challenge: In the beginning, the guard was too lazy. He would draw one giant box around a whole group of people (a "burst" of clicks) instead of boxing each person individually.
• The Fix: Christopher added a "post-processing" step. He used a mathematical tool (First Order Gradient) to find the exact "peaks" inside that giant box. It was like taking that giant box and slicing it up so that every single person got their own little box.

The Brain: The "Context" Detective

Here was the hardest part: How does the computer know if a sound is a Click or an Echo?

• The Problem: An echo sounds almost exactly like the original click. It's like hearing your own voice in a canyon. If you only hear one sound, it's impossible to tell if it's you or the canyon.
• The Human Trick: Biologists know that clicks usually come in a rhythmic pattern (like a drumbeat), while echoes are just random reflections.
• The AI Trick: Christopher taught a second AI, called a Random Forest, to be a "Context Detective." Instead of looking at one sound in isolation, this detective looked at the neighborhood.
  • Question: "Did a click happen 2 milliseconds ago?"
  • Question: "Is this sound weaker than the one before it?"
  • Question: "Does the pattern look like a rhythm?"
  • Verdict: If the pattern fits, it's a Click. If it's just a random bounce, it's an Echo.

The Results: From "Maybe" to "Mostly Right"

Christopher tested his system, which he named CLICK-SPOT, against the old methods:

• Old Methods (PAMGuard): Like a metal detector that beeps at every soda can and every coin. It found 39% of the real clicks correctly but was very confused.
• The New System (CLICK-SPOT): It found 96% of the clicks correctly and could tell the difference between a click and an echo with about 82% accuracy.

Why This Matters

Before this, scientists had to spend years manually labeling data. Now, they can process that data much faster.

• The Catch: The computer is currently slow. It takes 25 minutes of computer time to analyze just 1 minute of audio. It's not fast enough to listen to whales live while they are swimming by.
• The Future: The goal is to make it faster so it can be used on a boat in real-time. Also, because the system learns to recognize "click shapes," it could potentially be retrained to listen to dolphins, sperm whales, or even other animals that use sonar.

In a nutshell: Christopher built a digital pair of eyes and a smart brain that can look at a messy underwater recording, find the tiny "pings" of killer whales, and figure out which ones are the whales talking and which ones are just echoes, saving scientists thousands of hours of work.

Technical Summary

1. Problem Statement

The core challenge addressed in this thesis is the automated detection and classification of killer whale (Orcinus orca) echolocation clicks and their returning echoes in underwater bioacoustic recordings.

• Manual Limitations: Manual annotation by bioacousticians is extremely time-consuming (taking ~12 hours to label 1 minute of data containing bursts of clicks) and unsustainable for large-scale behavioral studies.
• Technical Challenges:
  • Low Signal-to-Noise Ratio (SNR): Clicks often occur in noisy environments with varying distances from the hydrophone.
  • Click vs. Echo Differentiation: Clicks and echoes share similar acoustic characteristics (frequency, duration). While echoes often have an inverted phase, this is not always reliable due to noise and reverberation.
  • Resolution Trade-offs: Traditional Short-Time Fourier Transform (STFT) spectrograms suffer from the uncertainty principle, offering a trade-off between time and frequency resolution that makes it difficult to isolate millisecond-scale Dirac-like impulses.
  • Context Dependency: Differentiating a click from an echo often requires analyzing the context (e.g., inter-arrival times in a burst) rather than isolated events.

2. Methodology

The thesis proposes a novel toolchain named CLICK-SPOT, which integrates image-based object detection with mathematical signal processing and ensemble learning. The methodology evolved through several stages:

A. Data Representation (SCWTSPEC)

Instead of using standard spectrograms alone, the author developed a multi-channel image representation called SCWTSPEC (Signal, Continuous Wavelet Transform, Spectrogram).

• Input: 5ms audio windows (960 samples at 192kHz).
• Channels:
  • Red: Raw Waveform.
  • Green: Continuous Wavelet Transform (CWT) using a Mexican Hat wavelet. This provides superior time resolution for high frequencies (where clicks occur) compared to STFT.
  • Blue: Spectrogram.
• Format: These are stacked into a 960x960x3 RGB image, allowing Convolutional Neural Networks (CNNs) to process all signal features simultaneously.

B. Stage 1: Event Detection (YOLO)

The thesis utilizes YOLO (You Only Look Once), specifically the YOLOv8 architecture, for object detection.

• Role: Detects bounding boxes around potential click/echo events within the SCWTSPEC images.
• Advantage: Unlike window-based classifiers (like ANIMAL-SPOT) that group events, YOLO can localize multiple distinct events within a single input window.

C. Stage 2: Post-Processing (First Order Detection - FOD)

YOLO often produces overlapping bounding boxes for single events or merged boxes for click-echo pairs. To refine this:

• FOD Algorithm: A First Order Gradient conversion is applied to the audio signal within the YOLO bounding boxes.
• Mechanism: It calculates the gradient of the signal to identify sharp Dirac-like peaks.
• Outcome: The FOD algorithm splits merged bounding boxes into distinct peaks, accurately defining the start and end times of individual clicks and echoes.

D. Stage 3: Classification (Random Forest)

YOLO detects events but cannot inherently distinguish between a Click and an Echo with high accuracy due to their acoustic similarity.

• Approach: A Random Forest classifier is applied as a post-processing step.
• Features: The classifier uses contextual features derived from the sequence of events, including:
  • Inter-arrival time between events.
  • Energy differences (min, max, mean).
  • FOD peak direction (phase).
  • Confidence scores from YOLO.
• Logic: By analyzing a window of events (e.g., 4 past + 1 current), the Random Forest learns the burst patterns typical of clicks versus the isolated nature of echoes.

3. Key Contributions

1. CLICK-SPOT Toolchain: The development of a fully automated pipeline combining YOLO (detection), FOD (refinement), and Random Forest (classification) specifically for cetacean echolocation.
2. SCWTSPEC Image Representation: A novel method of encoding audio data into 3-channel images (Waveform, CWT, Spectrogram) to leverage deep learning for high-frequency, short-duration signals.
3. Context-Aware Classification: Moving beyond isolated event detection by using a Random Forest to incorporate temporal context (inter-arrival times) to solve the click/echo differentiation problem.
4. Comparative Analysis: A rigorous evaluation of multiple approaches (PAMGuard, standalone FOD, ANIMAL-SPOT, and YOLO) demonstrating why traditional methods fail and why the proposed hybrid approach succeeds.

4. Experimental Results

The system was trained and tested on a dataset provided by Ocean Sounds e.V., consisting of Norwegian killer whale recordings (approx. 3 minutes of training data with ~7,000 annotations and a 2-minute evaluation set).

• Baseline Comparisons:
  • PAMGuard: Achieved only 39.7% overall accuracy due to high false positives and inability to differentiate clicks/echoes.
  • Standalone FOD: Improved accuracy to 53.1% but still suffered from false positives.
  • ANIMAL-SPOT: Achieved 63.9% accuracy but failed to isolate individual events due to its window-based architecture (grouping multiple clicks into one block).
• CLICK-SPOT Performance:
  • Event Detection: With FOD post-processing, YOLO achieved 79.53% precision and 89.81% recall for event detection.
  • Click Classification: The Random Forest classifier achieved a 95.93% accuracy in labeling clicks (distinguishing them from echoes and noise).
  • Overall Accuracy: The final toolchain achieved an overall click detection accuracy of 82.56%.
  • Correlation: The detected click rate showed a 98.01% correlation with the ground truth annotations, which was the primary metric for the biologist's use case (behavioral activity tracking).

5. Significance and Future Work

• Scientific Impact: CLICK-SPOT provides a scalable solution for analyzing massive bioacoustic datasets, enabling researchers to study killer whale social behavior and communication patterns without the bottleneck of manual labeling.
• Generalizability: While designed for killer whales, the architecture (Dirac-like pulse detection) is adaptable to other species (tested on pilot whales, sperm whales, and dolphins) via transfer learning.
• Limitations & Future Directions:
  • Real-Time Processing: The current system has a real-time factor of 25x (25 minutes to process 1 minute of audio), making it unsuitable for immediate field deployment without optimization.
  • Future Work: Integrating Recurrent Neural Networks (RNNs) with YOLO for better temporal context, reducing input image dimensions for speed, and refining the differentiation between Low, High, and Ultrasonic frequency clicks.

In conclusion, this thesis successfully demonstrates that combining advanced wavelet transformations with state-of-the-art object detection and contextual ensemble learning offers a superior solution for complex bioacoustic event detection compared to traditional threshold-based or single-model deep learning approaches.