arXiv:2506.10474cs.CV2025-06被引 3

大模型检测伪造图像能力有限,仍需人工辅助。

LLMs Are Not Yet Ready for Deepfake Image Detection

  • 用四款主流视觉语言模型零样本测试三类伪造图像
  • 准确率不高,易受复古风格等误导性视觉特征影响
  • 适合做辅助分析,提升专家判断的可解释性

深度伪造技术日益复杂,严重威胁媒体真实性和公众信任。与此同时,具备视觉推理能力的大规模视觉-语言模型(VLMs)在多个领域展现出潜力,引发对其用于深度伪造检测应用的关注。本研究对四种代表性VLMs(ChatGPT、Claude、Gemini、Grok)进行了结构化零样本评估,聚焦于人脸替换、重演和合成生成三类常见伪造形式。基于一个精心构建的基准数据集,涵盖来自多种来源的真实与篡改图像,评估各模型的分类准确率与推理深度。结果显示,尽管这些模型能生成连贯解释并识别表面异常,但尚不足以作为独立可靠的检测系统。研究揭示了关键失败模式:过度关注风格特征,且易受复古美学等误导性视觉模式影响。然而,它们在可解释性和上下文分析方面表现出优势,表明其在法证工作流中可作为人类专家的增强工具。因此,虽然通用模型当前尚不具备自主检测所需可靠性,但在混合或人机协同框架中具有重要应用前景。

原文摘要 · Abstract (English)

The growing sophistication of deepfakes presents substantial challenges to the integrity of media and the preservation of public trust. Concurrently, vision-language models (VLMs), large language models enhanced with visual reasoning capabilities, have emerged as promising tools across various domains, sparking interest in their applicability to deepfake detection. This study conducts a structured zero-shot evaluation of four prominent VLMs: ChatGPT, Claude, Gemini, and Grok, focusing on three primary deepfake types: faceswap, reenactment, and synthetic generation. Leveraging a meticulously assembled benchmark comprising authentic and manipulated images from diverse sources, we evaluate each model's classification accuracy and reasoning depth. Our analysis indicates that while VLMs can produce coherent explanations and detect surface-level anomalies, they are not yet dependable as standalone detection systems. We highlight critical failure modes, such as an overemphasis on stylistic elements and vulnerability to misleading visual patterns like vintage aesthetics. Nevertheless, VLMs exhibit strengths in interpretability and contextual analysis, suggesting their potential to augment human expertise in forensic workflows. These insights imply that although general-purpose models currently lack the reliability needed for autonomous deepfake detection, they hold promise as integral components in hybrid or human-in-the-loop detection frameworks.

深度伪造视觉语言模型可解释性检测

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。