用关键可疑区域推理，零微调实现跨域假图检测
Evidence Packing for Cross-Domain Image Deepfake Detection with LVLMs
- 从图像中提取可疑局部块，用语义与异常特征联合评分
- 在多个数据集上超越现有方法，无需微调大模型
- 适合需要快速部署、应对新伪造技术的检测场景
图像深度伪造检测（IDD）通过识别合成或篡改痕迹来区分真实与伪造图像。尽管大视觉语言模型（LVLM）具备强大图像理解能力，但将其应用于IDD通常需昂贵微调，且对多样、不断演化的伪造手法泛化能力差。本文提出语义一致证据包（SCEP），一种无需训练的LVLM框架，将全图推理替换为基于证据的推理。SCEP挖掘一组紧凑的可疑局部块令牌，以视觉编码器的CLS令牌作为全局参考，将局部特征聚类成语义连贯组，并使用融合指标对局部块打分，该指标结合了CLS引导的语义不一致性以及频率和噪声相关异常。为覆盖分散痕迹并避免冗余，SCEP每组采样少数高置信度局部块，并采用基于网格的非极大值抑制（NMS），生成一个证据包，用于条件化冻结的LVLM进行预测。在多个基准测试上的实验表明，SCEP在不进行LVLM微调的情况下优于强基线方法。
原文摘要 · Abstract (English)
Image Deepfake Detection (IDD) separates manipulated images from authentic ones by spotting artifacts of synthesis or tampering. Although large vision-language models (LVLMs) offer strong image understanding, adapting them to IDD often demands costly fine-tuning and generalizes poorly to diverse, evolving manipulations. We propose the Semantic Consistent Evidence Pack (SCEP), a training-free LVLM framework that replaces whole-image inference with evidence-driven reasoning. SCEP mines a compact set of suspicious patch tokens that best reveal manipulation cues. It uses the vision encoder's CLS token as a global reference, clusters patch features into coherent groups, and scores patches with a fused metric combining CLS-guided semantic mismatch with frequency-and noise-based anomalies. To cover dispersed traces and avoid redundancy, SCEP samples a few high-confidence patches per cluster and applies grid-based NMS, producing an evidence pack that conditions a frozen LVLM for prediction. Experiments on diverse benchmarks show SCEP outperforms strong baselines without LVLM fine-tuning.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。