用现代视觉大模型检测生成图像,实测效果远超专用工具。
Brought a Gun to a Knife Fight: Modern VFM Baselines Outgun Specialized Detectors on In-the-Wild AI Image Detection
- 用视觉基础模型+线性分类器,不需定制化设计。
- 真实场景准确率提升超20%,远超专用检测器。
- 适合关注实际应用与泛化能力的研究者。
尽管针对生成图像的专用检测器在受控基准上表现优异,但在真实世界场景中却因极高的漏检率而失效。我们不打造新的专用‘小刀’,而是引入‘枪’:基于现代视觉基础模型(VFM)的简单线性分类器。在相同数据上训练,该基线显著超越定制检测器,在真实场景中准确率提升超过20%。分析表明,VFM的强效源于其对合成图像与伪造相关概念(如‘AI-generated’)的语义对齐能力——这一对齐在最新版本如Perception Encoder、Meta CLIP2中已出现。我们进一步发现,这种能力依赖于预训练数据:当使用预训练截止日期后的新数据集测试时,对齐效果和整体性能均显著下降。因此得出两点关键结论:1)在真实世界对抗中,更新后的视觉基础模型的原始能力远胜于静态专用检测器;2)真正的泛化评估必须确保测试数据完全独立于模型整个训练历史,包括预训练阶段。
原文摘要 · Abstract (English)
While specialized detectors for AI-generated images excel on curated benchmarks, they fail catastrophically in real-world scenarios, as evidenced by their critically high false-negative rates on `in-the-wild' benchmarks. Instead of crafting another specialized `knife' for this problem, we bring a `gun' to the fight: a simple linear classifier on a modern Vision Foundation Model (VFM). Trained on identical data, this baseline decisively `outguns' bespoke detectors, boosting in-the-wild accuracy by a striking margin of over 20\%. Our analysis pinpoints the source of the VFM's `firepower': First, by probing text-image similarities, we find that recent VLMs (e.g., Perception Encoder, Meta CLIP2) have learned to align synthetic images with forgery-related concepts (e.g., `AI-generated'), unlike previous versions. Second, we speculate that this is due to data exposure, as both this alignment and overall accuracy plummet on a novel dataset scraped after the VFM's pre-training cut-off date, ensuring it was unseen during pre-training. Our findings yield two critical conclusions: 1) For the real-world `gunfight' of AI-generated image detection, the raw `firepower' of an updated VFM is far more effective than the `craftsmanship' of a static detector. 2) True generalization evaluation requires test data to be independent of the model's entire training history, including pre-training.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。