轻量级嵌套结构直接处理高维语音特征,提升防欺骗性能并降低计算开销
Nes2Net: A Lightweight Nested Architecture for Foundation Model Driven Speech Anti-spoofing
- 采用嵌套结构直接处理高维特征,避免降维层引入的参数和计算负担
- 在多个数据集上实现22%性能提升,后端计算成本降低87%
- 适合需要高效鲁棒语音反欺骗的工业部署场景
语音基础模型通过强大表征能力显著推动了各类语音任务的发展。然而,其高维输出特征常与下游任务模型所需的低维输入不匹配。现有方法通常引入降维层,但会增加参数量、计算开销,并可能丢失关键信息。为此,我们提出轻量级嵌套残差网络(Nes2Net),可直接处理高维特征,无需降维层。嵌套结构增强多尺度特征提取与交互,有效保留高维信息。在CtrSVDD歌声深度伪造检测数据集上,相比最先进基线,性能提升22%,后端计算成本降低87%。在ASVspoof 2021、ASVspoof 5、PartialSpoof和In-the-Wild四个多样化数据集上的广泛测试,涵盖全伪造语音、对抗攻击、部分伪造及真实场景,均验证了Nes2Net出色的鲁棒性与泛化能力。代码与预训练模型已开源。
原文摘要 · Abstract (English)
Speech foundation models have significantly advanced various speech-related tasks by providing exceptional representation capabilities. However, their high-dimensional output features often create a mismatch with downstream task models, which typically require lower-dimensional inputs. A common solution is to apply a dimensionality reduction (DR) layer, but this approach increases parameter overhead, computational costs, and risks losing valuable information. To address these issues, we propose Nested Res2Net (Nes2Net), a lightweight back-end architecture designed to directly process high-dimensional features without DR layers. The nested structure enhances multi-scale feature extraction, improves feature interaction, and preserves high-dimensional information. We first validate Nes2Net on CtrSVDD, a singing voice deepfake detection dataset, and report a 22% performance improvement and an 87% back-end computational cost reduction over the state-of-the-art baseline. Additionally, extensive testing across four diverse datasets: ASVspoof 2021, ASVspoof 5, PartialSpoof, and In-the-Wild, covering fully spoofed speech, adversarial attacks, partial spoofing, and real-world scenarios, consistently highlights Nes2Net's superior robustness and generalization capabilities. The code package and pre-trained models are available at https://github.com/Liu-Tianchi/Nes2Net.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。