Assessing AI-generated music detection in real-world broadcast monitoring

Technical Summary: Assessing AI-Generated Music Detection in Real-World Broadcast Monitoring

Problem Statement
The rapid proliferation of AI-generated music has created urgent challenges regarding transparency, authorship, and fair compensation in the broadcast industry. While existing AI-generated music detectors achieve near-perfect accuracy on isolated, high-fidelity tracks, they suffer from significant performance degradation when applied to real-world broadcast environments. Current evaluations are largely limited to synthetic data or controlled laboratory settings, failing to account for the acoustic complexities of television broadcasting. These complexities include short music segments, dominant speech or sound effects masking the music, variable loudness levels, and low-bitrate transmission constraints (e.g., 8 kHz sampling, 40 kbps AAC-LC encoding). Consequently, the reliability of current detection methods in actual broadcast monitoring remains unresolved.

Methodology
To address the gap between synthetic benchmarks and real-world conditions, the authors introduce BAMM (Broadcast AI-Music Monitoring), a novel dataset and evaluation framework.

• Dataset Construction (BAMM): BAMM consists of 40 hours of real-world television recordings, balanced between approximately 20 hours of AI-generated music and 20 hours of human-made music. The dataset was curated by:

  1. Reference Selection: Creating a corpus of clean reference tracks. Human-made tracks were restricted to releases between 2020–2022 (pre-dating advanced generative models like Suno v3.5). AI-generated tracks were selected only if they received unanimous positive classification from an ensemble of five diverse detectors.
  2. Fingerprinting: Using audio fingerprinting to locate these reference tracks within a global broadcast archive (monitoring 4,200+ channels) for airings between January 2025 and March 2026.
  3. Filtering: Applying a Deep Music Detector (DMD) to isolate clips containing music (foreground or background) and removing sound effects or false matches.
  4. Characteristics: Clips range from 5 to 60 seconds. The audio is monophonic, sampled at 8 kHz, and encoded at 40 kbps, representing the low-quality proxy streams used in industrial monitoring.

• Experimental Setup: The study evaluates two variants of a Convolutional Neural Network (CNN) architecture (based on Afchar et al. [6]) across three progressively challenging scenarios:

  1. Clean Foreground Music (CFM): High-quality, isolated tracks (control baseline).
  2. Synthetic TV Broadcast (STB): Controlled mixtures of music and speech simulating broadcast conditions (using the AI-OpenBMAT dataset).
  3. Real TV Broadcast (RTB): The full BAMM dataset containing authentic broadcast recordings.

  The two models compared are:

  • CNN Clean: Trained exclusively on clean, foreground music.
  • CNN Broadcast: Trained on a mix of clean music and synthetic broadcast conditions (music mixed with speech at varying Signal-to-Noise Ratios).

Key Contributions

1. BAMM Dataset: The primary contribution is the release of a 40-hour, real-world broadcast dataset specifically curated for AI-generated music detection. Unlike previous benchmarks, it captures the full variability of broadcast environments, including background music, sound effects, and low-bitrate encoding.
2. Empirical Evaluation: The paper provides a rigorous benchmark comparing "clean-trained" versus "broadcast-trained" models across synthetic and real broadcast scenarios.
3. Identification of Domain Gaps: The study systematically quantifies the performance drop when moving from clean audio to synthetic mixtures, and finally to real broadcast content, highlighting the insufficiency of current training approaches.

Results
The evaluation reveals a critical domain gap and substantial performance degradation as scenarios become more realistic:

• Clean Foreground Music (CFM): Both models achieve near-perfect performance (F1 > 0.99), confirming that the artifacts learned from low-quality training data are detectable in high-quality audio.
• Synthetic TV Broadcast (STB): Performance degrades significantly. The CNN Clean model drops to an F1 of 0.342, while the CNN Broadcast model improves to 0.661. This indicates that while training on mixed data helps, it does not fully resolve the masking effects of broadcast conditions.
• Real TV Broadcast (RTB): Performance deteriorates further on the BAMM dataset.
  • CNN Clean: F1 drops to 0.186.
  • CNN Broadcast: F1 drops to 0.472.
  • Score Overlap: There is substantial overlap in probability scores between AI-generated and human-made music, particularly for background music. The models struggle more with rejecting human content than with identifying AI content in these conditions.
  • Foreground vs. Background: Both models perform better on foreground music than background music, confirming that masking is a primary source of degradation. However, even foreground detection remains unreliable due to degraded audio quality and other broadcast sounds.

Significance and Claims
The paper claims that current training approaches for CNN-based detectors are insufficient for reliable AI-generated music detection in broadcast monitoring. The results expose a critical domain gap where models trained on clean or synthetically mixed data fail to generalize to the complex, low-fidelity reality of television broadcasts.

The authors position BAMM not as a solution to the detection problem, but as a necessary reference evaluation setting. By exposing failure modes not captured by clean-music datasets, BAMM highlights the need for more robust detection methods capable of modeling the acoustic and contextual complexity of real broadcast media. The work underscores that the task of detecting AI-generated music becomes significantly more difficult when the musical signal is less salient or embedded within complex audio mixtures, a challenge that current state-of-the-art architectures have not yet overcome.