现有语音伪造检测器难以识别仅用人脸图像生成的伪造语音。
Can Current Detectors Catch Face-to-Voice Deepfake Attacks?
- 系统评估了基于人脸生成语音的伪造技术检测效果。
- 主流检测器在干净和噪声环境下均表现不佳,准确率低于60%。
- 针对性微调可提升检测能力,但可能降低对其他伪造方法的泛化性。
生成模型的快速发展使得合成语音(即音频深度伪造)愈发隐蔽。最近的FOICE方法(USENIX'24)展示了一项令人担忧的能力:仅需一张人脸图像,即可生成目标人物的语音,无需任何原始语音样本。该方法利用面部与声学特征间的相关性,生成的语音足以绕过包括WeChat语音指纹和Microsoft Azure在内的行业标准认证系统。由于人脸图像比语音样本更易获取,这大幅降低了大规模攻击的门槛。本文研究两个核心问题:(RQ1) 当前最先进的音频深度伪造检测器能否在干净和噪声条件下可靠识别FOICE生成的语音?(RQ2) 在FOICE数据上微调是否能提升检测性能而不引发过拟合,从而保持对SpeechT5等未知生成器的鲁棒性?本研究有三项贡献:第一,首次系统评估了FOICE的检测效果,发现主流检测器在标准与噪声条件下均持续失效;第二,提出针对性微调策略,有效捕捉FOICE特有痕迹,显著提升准确率;第三,评估微调后的泛化能力,揭示在专精于FOICE与保持对未知生成器鲁棒性之间的权衡。这些发现暴露了当前防御体系的根本缺陷,推动下一代音频深度伪造检测架构与训练协议的发展。
原文摘要 · Abstract (English)
The rapid advancement of generative models has enabled the creation of increasingly stealthy synthetic voices, commonly referred to as audio deepfakes. A recent technique, FOICE [USENIX'24], demonstrates a particularly alarming capability: generating a victim's voice from a single facial image, without requiring any voice sample. By exploiting correlations between facial and vocal features, FOICE produces synthetic voices realistic enough to bypass industry-standard authentication systems, including WeChat Voiceprint and Microsoft Azure. This raises serious security concerns, as facial images are far easier for adversaries to obtain than voice samples, dramatically lowering the barrier to large-scale attacks. In this work, we investigate two core research questions: (RQ1) can state-of-the-art audio deepfake detectors reliably detect FOICE-generated speech under clean and noisy conditions, and (RQ2) whether fine-tuning these detectors on FOICE data improves detection without overfitting, thereby preserving robustness to unseen voice generators such as SpeechT5. Our study makes three contributions. First, we present the first systematic evaluation of FOICE detection, showing that leading detectors consistently fail under both standard and noisy conditions. Second, we introduce targeted fine-tuning strategies that capture FOICE-specific artifacts, yielding significant accuracy improvements. Third, we assess generalization after fine-tuning, revealing trade-offs between specialization to FOICE and robustness to unseen synthesis pipelines. These findings expose fundamental weaknesses in today's defenses and motivate new architectures and training protocols for next-generation audio deepfake detection.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。