120 papers
This paper evaluates the trade-off between privacy and Parkinson's disease detection in anonymized speech, demonstrating that while STT-TTS anonymization severely degrades diagnostic performance by erasing prosodic cues, kNN-VC effectively preserves macro-prosodic features to maintain high detection accuracy with only a minor performance drop.
This paper introduces VASR, a multimodal reasoning framework for Context-Aware Visual Speech Recognition (CAVSR) that leverages an Audio-Visual Chain-of-Thought (AV-CoT) to explicitly ground acoustic signals with rich visual context like scenes and on-screen text, thereby overcoming single-modality dominance and achieving state-of-the-art performance.
This paper introduces a comprehensive multimodal dataset comprising audio and vibration signals from a single-speed chain conveyor system, designed to benchmark robust industrial fault detection and classification under diverse operating conditions and noise levels through standardized evaluation protocols and baseline models.
This paper introduces Task 5 of the DCASE 2025 Challenge, a multi-domain Audio Question Answering benchmark designed to evaluate and advance the acoustic reasoning capabilities of audio-language models across diverse scenarios including bioacoustics, temporal soundscapes, and complex real-world clips.
This paper proposes a zero-AV-resource framework for audiovisual speech recognition that generates synthetic talking-head videos by lip-syncing static facial images with real audio, successfully enabling high-performance model training for under-resourced languages like Catalan without the need for labeled video corpora.
This paper introduces the Cross-Lingual Transfer Matrix (CLTM) to systematically quantify language-dependent performance variations in paralinguistic tasks like gender identification and speaker verification, revealing that despite their acoustic nature, these tasks exhibit distinct cross-lingual transfer patterns when using multilingual HuBERT-based encoders.
DualTurn is a dual-channel generative speech model that learns natural turn-taking dynamics through unsupervised pretraining on conversational audio and fine-tuning to predict agent actions, outperforming existing methods in both action prediction accuracy and turn-boundary anticipation while enabling tool-calling capabilities.
This chapter reviews recent computational models demonstrating that self-supervised and visually grounded learning principles can effectively explain early language acquisition from acoustic and audiovisual speech without relying on strong linguistic priors.
This paper demonstrates that scaling self-supervised speech models from 1,000 to 4,000 languages triggers a non-linear shift that enables the discovery of deep genealogical relationships and complex contact patterns, exemplified by the emergence of a robust Pacific macro-cluster driven by shared acoustic signatures.
The TCG CREST system achieved a sixth-place ranking in the DISPLACE-M challenge's speaker diarization track by demonstrating that a hybrid end-to-end Diarizen framework with WavLM embeddings and optimized agglomerative hierarchical clustering significantly outperformed a SpeechBrain baseline, reducing the diarization error rate to 9.21% on the evaluation set.
LongAudio-RAG is a hybrid edge-cloud framework that enables precise, low-hallucination question answering over multi-hour audio streams by converting recordings into timestamped event records for SQL-based retrieval, which then grounds Large Language Model responses in structured evidence rather than raw audio.
The paper introduces ECHO, a novel foundation model that leverages band-split architecture and frequency positional embeddings to achieve state-of-the-art performance in anomaly detection and fault classification across variable-length, arbitrary sampling rate machine signals without requiring padding or cropping.
BemaGANv2 is an advanced GAN-based vocoder that enhances long-term audio generation for Text-to-Music and Text-to-Audio applications by integrating Anti-aliased Multi-Periodicity composition modules in the generator and systematically evaluating novel discriminator combination strategies, including the Multi-Envelope Discriminator, to achieve high-fidelity and temporally coherent results.
This paper introduces Trilobyte, a byte-level tokenization schema that enables tractable lossless compression of full-fidelity (up to 24-bit) audio using autoregressive language models, demonstrating that while these models outperform FLAC at lower bit depths, their compression gains diminish as bit depth increases.
FoleyFlow introduces a novel video-to-audio generation framework that achieves superior semantic and rhythmic synchronization by aligning unimodal encoders through masked audio-visual modeling and employing a dynamic conditional flow that utilizes temporally varying video features to guide audio synthesis.
This paper introduces an analysis-driven framework that generates a publicly available, 19-hour procedural engine sound dataset with sample-accurate RPM and torque annotations by extracting harmonic structures from real recordings to drive a parametric synthesizer, thereby addressing the scarcity of clean, standardized audio data for automotive sound design and machine learning applications.
This paper proposes a lightweight, self-supervised framework that augments a frozen speech enhancement backbone with low-rank adapters, enabling efficient on-device adaptation to dynamic real-world noise conditions by updating fewer than 1% of parameters while achieving significant signal quality improvements.
This paper proposes a fast and flexible Vocos-based bandwidth extension model that generates missing high-frequency audio content up to 48 kHz using a single neural network and a lightweight refiner, achieving competitive quality with extreme real-time throughput on both GPU and CPU hardware.
This study presents an automated pipeline using a wearable SonicGuard sensor and a pretrained Audio Spectrogram Transformer to accurately segment and classify bowel sounds, significantly reducing manual labeling time while providing clinicians with an objective, quantitative tool for assessing gastrointestinal function.
This paper demonstrates that four recent deep speech denoising models are vulnerable to psychoacoustically hidden adversarial noise, which can render their output unintelligible while remaining imperceptible to human listeners, thereby highlighting critical safety concerns for their deployment in high-stakes applications.