简单特征融合比复杂注意力更有效，提升语音活动检测精度与效率
Attention Is Not Always the Answer: Optimizing Voice Activity Detection with Simple Feature Fusion
- 用拼接、相加、交叉注意力三种方式融合传统特征与预训练模型特征
- 相加融合在多个数据集上超越当前最佳模型，平均准确率提升2.04%
- 适合需要高效高鲁棒性语音检测的场景，如实时通信和嵌入式设备
语音活动检测（VAD）在语音处理中至关重要，通常依赖手工特征或神经网络特征。本文研究了梅尔频率倒谱系数（MFCCs）与多种预训练模型（wav2vec 2.0、HuBERT、WavLM、UniSpeech、MMS、Whisper）特征的有效性。提出FusionVAD统一框架，采用拼接、相加、交叉注意力三种融合策略。实验表明，简单融合方法（尤其是相加）在准确率和效率上均优于交叉注意力。基于融合的模型持续优于单一特征模型，证明了两类特征具有互补性。最佳融合模型在多个数据集上超越当前最优的Pyannote，平均绝对提升2.04%。结果证实，简单特征融合可增强VAD鲁棒性并保持计算高效。
原文摘要 · Abstract (English)
Voice Activity Detection (VAD) plays a key role in speech processing, often utilizing hand-crafted or neural features. This study examines the effectiveness of Mel-Frequency Cepstral Coefficients (MFCCs) and pre-trained model (PTM) features, including wav2vec 2.0, HuBERT, WavLM, UniSpeech, MMS, and Whisper. We propose FusionVAD, a unified framework that combines both feature types using three fusion strategies: concatenation, addition, and cross-attention (CA). Experimental results reveal that simple fusion techniques, particularly addition, outperform CA in both accuracy and efficiency. Fusion-based models consistently surpass single-feature models, highlighting the complementary nature of MFCCs and PTM features. Notably, our best-performing fusion model exceeds the state-of-the-art Pyannote across multiple datasets, achieving an absolute average improvement of 2.04%. These results confirm that simple feature fusion enhances VAD robustness while maintaining computational efficiency.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。