用音频大模型识别假音频，尤其在数据少时表现更好
ALLM4ADD: Unlocking the Capabilities of Audio Large Language Models for Audio Deepfake Detection
- 将音频真伪判断转为问答任务，用提示词引导大模型分析
- 在数据稀缺场景下，检测准确率超越传统方法
- 首个探索音频大模型用于深度伪造检测的框架
由于高保真音频生成模型的兴起及其潜在滥用风险，音频深度伪造检测（ADD）日益重要。尽管音频大语言模型（ALLM）在多种音频处理任务中取得进展，但其在ADD中的适用性尚不明确。本文首次对ALLM在ADD任务上的零样本能力进行系统评估，发现其表现不佳。为此，提出ALLM4ADD框架：将ADD重构为音频问答任务，通过提问“该音频是伪造还是真实？”引导模型判断。经监督微调后，模型可有效评估音频真实性。大量实验表明，该方法在数据稀缺场景下仍能实现优异性能，显著优于基线。本研究为利用大模型构建更高效音频伪造检测系统提供新思路。代码已开源。
原文摘要 · Abstract (English)
Audio deepfake detection (ADD) has grown increasingly important due to the rise of high-fidelity audio generative models and their potential for misuse. Given that audio large language models (ALLMs) have made significant progress in various audio processing tasks, a heuristic question arises: \textit{Can ALLMs be leveraged to solve ADD?}. In this paper, we first conduct a comprehensive zero-shot evaluation of ALLMs on ADD, revealing their ineffectiveness. To this end, we propose ALLM4ADD, an ALLM-driven framework for ADD. Specifically, we reformulate ADD task as an audio question answering problem, prompting the model with the question: ``Is this audio fake or real?''. We then perform supervised fine-tuning to enable the ALLM to assess the authenticity of query audio. Extensive experiments are conducted to demonstrate that our ALLM-based method can achieve superior performance in fake audio detection, particularly in data-scarce scenarios. As a pioneering study, we anticipate that this work will inspire the research community to leverage ALLMs to develop more effective ADD systems. Code is available at https://github.com/ucas-hao/qwen_audio_for_add.git
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。