构建大规模可控语音活动检测数据集,推动模型在复杂环境下的泛化能力。
LibriVAD: A Scalable Open Dataset with Deep Learning Benchmarks for Voice Activity Detection
- 基于LibriSpeech合成带噪语音,可控信噪比与静音比例
- ViT+MFCC在多种场景下超越传统模型,尤其在未知噪声中表现优异
- 提供三种规模数据集和开源代码,适合语音处理与鲁棒性研究者
稳健的语音活动检测(VAD)在嘈杂、多变且未见的声学条件下仍具挑战性。当前研究的一大瓶颈是缺乏大规模、系统可控且公开的数据集。为此,我们提出LibriVAD——一个从LibriSpeech衍生并融合真实与合成噪声的可扩展开源数据集。该数据集支持对语音-噪声比、静音-语音比(SSR)及噪声多样性进行系统控制,并以15 GB、150 GB、1.5 TB三个规模发布,含两种变体(LibriVAD-NonConcat与LibriVAD-Concat),适配不同实验需求。我们评估了多种特征-模型组合,包括波形、梅尔频率倒谱系数(MFCC)与伽马振荡器滤波器组倒谱系数,并首次将视觉变换器(ViT)引入VAD任务。实验表明,采用MFCC特征的ViT在已见、未见及分布外(OOD)条件下均显著优于增强型深度神经网络与卷积长短期记忆深度神经网络,包括在真实场景VOiCES数据集上的测试。进一步分析显示,扩大数据规模并平衡SSR能显著提升模型在OOD条件下的泛化性能。所有数据集、训练模型与代码均已开源,以促进可复现性并加速VAD研究进展。
原文摘要 · Abstract (English)
Robust Voice Activity Detection (VAD) remains a challenging task, especially under noisy, diverse, and unseen acoustic conditions. Beyond algorithmic development, a key limitation in advancing VAD research is the lack of large-scale, systematically controlled, and publicly available datasets. To address this, we introduce LibriVAD - a scalable open-source dataset derived from LibriSpeech and augmented with diverse real-world and synthetic noise sources. LibriVAD enables systematic control over speech-to-noise ratio, silence-to-speech ratio (SSR), and noise diversity, and is released in three sizes (15 GB, 150 GB, and 1.5 TB) with two variants (LibriVAD-NonConcat and LibriVAD-Concat) to support different experimental setups. We benchmark multiple feature-model combinations, including waveform, Mel-Frequency Cepstral Coefficients (MFCC), and Gammatone filter bank cepstral coefficients, and introduce the Vision Transformer (ViT) architecture for VAD. Our experiments show that ViT with MFCC features consistently outperforms established VAD models such as boosted deep neural network and convolutional long short-term memory deep neural network across seen, unseen, and out-of-distribution (OOD) conditions, including evaluation on the real-world VOiCES dataset. We further analyze the impact of dataset size and SSR on model generalization, experimentally showing that scaling up dataset size and balancing SSR noticeably and consistently enhance VAD performance under OOD conditions. All datasets, trained models, and code are publicly released to foster reproducibility and accelerate progress in VAD research.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。