用深度学习检测环境音效伪造，准确率达98%
Environmental Sound Deepfake Detection Using Deep-Learning Framework
- 分两阶段微调预训练BEATs模型，提升检测精度
- 在基准数据集上达98%准确率，跨数据集仍保持86%准确
- 首次区分音景与声音事件的伪造检测任务
本文提出一种用于环境音效深度伪造检测（ESDD）的深度学习框架，旨在判断输入音频中音景与声音事件是否为伪造。通过大量实验分析时频图、网络结构及预训练模型对性能的影响，结果表明音景与声音事件的伪造检测应视为独立任务。研究发现，微调预训练模型比从零开始训练更有效。最优模型采用提出的两阶段训练策略微调BEATs模型，在EnvSDD测试集上取得0.98的准确率、0.95的F1值和0.99的AUC值；在ESD-Challenge-TestSet跨数据集测试中，准确率为0.86，F1值为0.80，AUC为0.93。
原文摘要 · Abstract (English)
In this paper, we propose a deep-learning framework for Environmental Sound Deepfake Detection (ESDD) - the task of identifying whether the sound scene and sound event in an input audio recording is fake or real. To this end, we first conduct extensive experiments to explore how individual spectrograms, a wide range of network architectures, and pre-trained models affect the performance of an ESDD model. The experimental results on the benchmark datasets of EnvSDD indicate that detecting deepfake audio of sound scenes and detecting deepfake audio of sound events should be considered as individual tasks. We also show that fine-tuning a pre-trained model is more effective than training a model from scratch for ESDD. Ultimately, our best model, which fine-tunes the pre-trained BEATs model using the proposed two-phase training strategy, achieves an Accuracy of 0.98, F1 score of 0.95, and AUC score of 0.99 on the Test subset of the EnvSDD dataset. Our best model also achieves an Accuracy of 0.86, F1 score of 0.80, and AUC of 0.93 when evaluated cross-dataset on the ESD-Challenge-TestSet dataset.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。