用轻量网络和对比学习检测深度伪造语音,效果优于竞赛优胜方案。
DIN-CTS: Low-Complexity Depthwise-Inception Neural Network with Contrastive Training Strategy for Deepfake Speech Detection
- 设计轻量级深度可分离卷积的Inception网络,结合对比学习训练。
- 在4秒音频上达到4.6%误报率,参数仅177万,计算量985兆次。
- 适合实时部署,尤其适用于资源受限设备上的语音真伪检测。
本文提出一种基于低复杂度深度可分离Inception网络(DIN)与对比学习策略(CTS)的深度伪造语音检测方法。输入音频经短时傅里叶变换(STFT)与线性滤波器(LF)转换为频谱图后,用于训练DIN。训练完成后,DIN提取真实语音的音频嵌入,构建高斯分布模型;测试时通过计算待测语音与该分布的距离判断其真伪。在ASVspoof 2019 LA基准数据集上实验表明,该系统在单模型下仅需1.77M参数和985M FLOPS,对4秒短音频实现4.6%的等错误率(EER)、95.4%准确率(Acc.)、97.3% F1值和98.9% AUC,超越该挑战赛单系统最优提交结果,具备实时应用潜力。
原文摘要 · Abstract (English)
In this paper, we propose a deep neural network approach for deepfake speech detection (DSD) based on a lowcomplexity Depthwise-Inception Network (DIN) trained with a contrastive training strategy (CTS). In this framework, input audio recordings are first transformed into spectrograms using Short-Time Fourier Transform (STFT) and Linear Filter (LF), which are then used to train the DIN. Once trained, the DIN processes bonafide utterances to extract audio embeddings, which are used to construct a Gaussian distribution representing genuine speech. Deepfake detection is then performed by computing the distance between a test utterance and this distribution to determine whether the utterance is fake or bonafide. To evaluate our proposed systems, we conducted extensive experiments on the benchmark dataset of ASVspoof 2019 LA. The experimental results demonstrate the effectiveness of combining the Depthwise-Inception Network with the contrastive learning strategy in distinguishing between fake and bonafide utterances. We achieved Equal Error Rate (EER), Accuracy (Acc.), F1, AUC scores of 4.6%, 95.4%, 97.3%, and 98.9% respectively using a single, low-complexity DIN with just 1.77 M parameters and 985 M FLOPS on short audio segments (4 seconds). Furthermore, our proposed system outperforms the single-system submissions in the ASVspoof 2019 LA challenge, showcasing its potential for real-time applications.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。