arXiv:2509.25502cs.CV2025-09被引 25

让AI先看清假图痕迹,再推理判断,提升检测准确性和解释力。

Seeing Before Reasoning: A Unified Framework for Generalizable and Explainable Fake Image Detection

  • 先训练模型感知伪造痕迹,再进行推理,解决视觉敏感度不足问题。
  • 在多个数据集上检测准确率提升,且解释内容与真实伪造特征高度相关。
  • 适合需要可解释性检测的场景,如社交媒体审核、司法取证。

利用多模态大语言模型(MLLM)检测AI生成图像受到越来越多关注,因其具备丰富世界知识、常识推理能力及潜在可解释性。然而,直接应用现有MLLM进行检测常表现不佳。我们指出,根本原因在于:模型在真正‘看见’前就被要求‘推理’。首先,其视觉编码器主要优化于语义识别,对细微伪造痕迹不敏感;其次,微调数据采用狭隘指令格式,与预训练分布差异大,导致模型依赖语言捷径,遗忘预训练知识。为此,我们提出新范式:先‘看’后‘想’。构建Forensic-Chat系统,强化模型对伪造痕迹的感知能力,实现通用、可解释且支持多轮对话的假图检测。同时提出ExplainFake-Bench基准,从五个维度评估解释质量。大量实验表明,该方法在泛化性和可信解释方面显著优于现有方案。

原文摘要 · Abstract (English)

Detecting AI-generated images with multimodal large language models (MLLMs) has gained increasing attention, due to their rich world knowledge, common-sense reasoning, and potential for explainability. However, naively applying those MLLMs for detection often leads to suboptimal performance. We argue that the root of this failure lies in a fundamental mismatch: MLLMs are asked to reason about fakes before they can truly see them. First, they do not really see: existing MLLMs' vision encoders are primarily optimized for semantic-oriented recognition rather than the perception of low-level signals, leaving them insensitive to subtle forgery traces. Without access to reliable perceptual evidence, the model grounds its judgment on incomplete and limited visual observations. Second, existing finetuning data for detection typically uses narrow, instruction-style formats, which diverge sharply from the diverse, heterogeneous distributions seen in pretraining. In the absence of meaningful visual cues, the model therefore exploits these linguistic shortcuts, resulting in catastrophic forgetting of pretrained knowledge (even the basic dialogue capabilities). In response, we advocate for a new paradigm: seeing before reasoning. We propose that MLLMs should first be trained to perceive artifacts-strengthening their artifact-aware visual perception-so that subsequent reasoning is grounded in actual observations. We therefore propose Forensic-Chat, a generalizable, explainable, and still-conversational (for multi-round dialogue) assistant for fake image detection. We also propose ExplainFake-Bench, a benchmark tailored for the evaluation of the MLLM's explainability for image forensics from five key aspects. Extensive experiments show its superiority of generalization and genuinely reliable explainability.

假图检测可解释性多模态视觉感知

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。