用少量失败样本实现无需训练的通用假图像检测
Leveraging Failed Samples: A Few-Shot and Training-Free Framework for Generalized Deepfake Detection

- 仅需一个假图像样本,通过最近邻比对完成检测
- 在29个生成模型上平均提升8.7%性能
- 适合真实场景中新增假图快速响应的检测需求
近期深度伪造检测研究通常将未知样本检测视为零样本任务,即在已知模型生成图像上训练,但泛化到未知模型。然而,当模型对未知样本表现不佳时,这些样本仍可获取并分析,表明应视为少样本任务。本文提出少样本无训练网络(FTNet),用于真实世界中的少样本深度伪造检测。不同于依赖大规模已知数据训练的传统方法,FTNet仅使用评估集中一个伪造样本,模拟现实中新样本出现后可收集利用的场景,无需训练或参数更新。测试时,每个样本与已知真伪样本比较,依据最近邻类别分类。我们在29个不同生成模型的AI生成图像上进行综合分析,达到新最优性能,相比现有方法平均提升8.7%。该工作为真实世界深度伪造检测提供了新视角:当模型在少样本上泛化失败时,利用这些失败样本反而能提升性能。
原文摘要 · Abstract (English)
Recent deepfake detection studies often treat unseen sample detection as a ``zero-shot" task, training on images generated by known models but generalizing to unknown ones. A key real-world challenge arises when a model performs poorly on unknown samples, yet these samples remain available for analysis. This highlights that it should be approached as a ``few-shot" task, where effectively utilizing a small number of samples can lead to significant improvement. Unlike typical few-shot tasks focused on semantic understanding, deepfake detection prioritizes image realism, which closely mirrors real-world distributions. In this work, we propose the Few-shot Training-free Network (FTNet) for real-world few-shot deepfake detection. Simple yet effective, FTNet differs from traditional methods that rely on large-scale known data for training. Instead, FTNet uses only one fake samplefrom an evaluation set, mimicking the scenario where new samples emerge in the real world and can be gathered for use, without any training or parameter updates. During evaluation, each test sample is compared to the known fake and real samples, and it is classified based on the category of the nearest sample. We conduct a comprehensive analysis of AI-generated images from 29 different generative models and achieve a new SoTA performance, with an average improvement of 8.7\% compared to existing methods. This work introduces a fresh perspective on real-world deepfake detection: when the model struggles to generalize on a few-shot sample, leveraging the failed samples leads to better performance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。