通过双分支结构分离全局与局部特征,提升语音深度伪造检测精度。
Disentangled Global-Local Feature Learning with E-Branchformer for Audio Deepfake Detection

- 双分支并行处理:自注意力捕捉全局上下文,卷积提取局部时序模式。
- 在ASVspoof 2021 LA等数据集上达到0.88%~6.30%的最低误判率。
- 适合需要高鲁棒性的真实场景语音安全系统开发者使用。
语音合成技术如文本转语音和语音转换的快速发展对基于语音的认证系统构成严重威胁,亟需可靠的深度伪造检测方法。本文提出一种基于E-Branchformer的新架构,有效利用自监督语音表征进行音频深度伪造检测。模型采用并行分支,通过多头自注意力捕捉全局上下文依赖,同时用卷积处理提取局部时序模式。为增强判别能力,融合深度可分离卷积与挤压激励模块,在特征融合后丰富分类令牌的信息。在ASVspoof 2021 LA、DF和In-the-Wild数据集上的大量实验表明,该模型取得0.88%、1.85%和6.30%的等错误率,显著优于现有方法。消融实验证实双分支架构提供互补判别信息,挤压激励聚合显著提升自监督特征融合效果,且DWConv与SE模块组合对分类令牌增强至关重要。在真实场景下的优异表现证明其对多样声学条件和未知欺骗攻击具有强泛化能力。
原文摘要 · Abstract (English)
The rapid advancement of voice synthesis technologies such as text-to-speech and voice conversion poses significant threats to speech-based authentication systems, necessitating robust deepfake detection methods. In this work, we propose a novel E-Branchformer-based architecture that effectively leverages self-supervised speech representations for audio deepfake detection. Our model employs parallel branches to simultaneously capture global contextual dependencies through multi-head self-attention and local temporal patterns through convolutional processing. To enhance discriminative capability, we integrate depthwise convolution and Squeeze-and-Excitation modules that enrich the classification token with refined patch token information after feature merging. Extensive experiments on ASVspoof 2021 LA, DF, and In-the-Wild datasets demonstrate state-of-the-art performance with equal error rates of 0.88%, 1.85%, and 6.30% respectively, substantially outperforming existing methods. Comprehensive ablation studies validate that the dual-branch architecture provides complementary discriminative information, Squeeze-and-Excitation Aggregation significantly improves SSL feature integration, and the combination of DWConv and SE modules is critical for effective class token enhancement. The superior performance on real-world scenarios demonstrates strong generalization capability to diverse acoustic conditions and unseen spoofing attacks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。