直接处理原始音频波形,提升语音伪造检测的泛化能力。
End-to-end Audio Deepfake Detection from RAW Waveforms: a RawNet-Based Approach with Cross-Dataset Evaluation
- 基于RawNetLite的端到端模型,无需人工特征提取
- 在真实场景下实现83.4%准确率,误报率仅16.4%
- 融合编码器干扰与波形增强,适合实际部署
语音深度伪造对数字安全与信任构成日益严峻的威胁,利用先进生成模型制造出高度逼真的合成语音。在开放世界条件下,测试时遇到的伪造方法可能与训练阶段不同,检测难度显著增加。本文提出一种端到端深度学习框架,直接对原始波形进行音频伪造检测。所提模型RawNetLite为轻量级卷积-循环架构,可同时捕捉频谱与时间特征,无需手工预处理。为提升鲁棒性,采用多源数据混合训练,并引入焦点损失(Focal Loss)强化难样本学习。进一步表明,在训练中加入编码器级伪造数据并应用波形级音频增强(如音高偏移、噪声、时间拉伸),可显著提升在真实声学条件下的泛化性能。模型在域内数据集FakeOrReal上达到99.7% F1与0.25% EER,而在更具挑战性的跨分布测试集AVSpoof2021 + CodecFake上仍保持83.4% F1与16.4% EER。结果凸显多样化训练数据、定制目标函数及音频增强对构建稳健、通用语音伪造检测器的重要性。代码与预训练模型已开源:https://iplab.dmi.unict.it/mfs/Deepfakes/PaperRawNet2025/。
原文摘要 · Abstract (English)
Audio deepfakes represent a growing threat to digital security and trust, leveraging advanced generative models to produce synthetic speech that closely mimics real human voices. Detecting such manipulations is especially challenging under open-world conditions, where spoofing methods encountered during testing may differ from those seen during training. In this work, we propose an end-to-end deep learning framework for audio deepfake detection that operates directly on raw waveforms. Our model, RawNetLite, is a lightweight convolutional-recurrent architecture designed to capture both spectral and temporal features without handcrafted preprocessing. To enhance robustness, we introduce a training strategy that combines data from multiple domains and adopts Focal Loss to emphasize difficult or ambiguous samples. We further demonstrate that incorporating codec-based manipulations and applying waveform-level audio augmentations (e.g., pitch shifting, noise, and time stretching) leads to significant generalization improvements under realistic acoustic conditions. The proposed model achieves over 99.7% F1 and 0.25% EER on in-domain data (FakeOrReal), and up to 83.4% F1 with 16.4% EER on a challenging out-of-distribution test set (AVSpoof2021 + CodecFake). These findings highlight the importance of diverse training data, tailored objective functions and audio augmentations in building resilient and generalizable audio forgery detectors. Code and pretrained models are available at https://iplab.dmi.unict.it/mfs/Deepfakes/PaperRawNet2025/.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。