用视觉语言模型识别假图并溯源生成模型，准确率超95%
FIDAVL: Fake Image Detection and Attribution using Vision-Language Model
- 结合视觉与语言互补性，采用软提示调优实现多任务检测与溯源
- 假图检测平均准确率95.42%，源模型归属F1-score达92.64%
- 适用于生成式模型安全评估，适合关注深度伪造检测的研究者
我们提出FIDAVL：一种基于视觉语言模型的假图像检测与溯源方法。该方法受视觉与语言处理协同效应启发，利用零样本学习优势，结合视觉与语言的互补性及软提示调优策略，同时实现假图像检测与来源模型精准归属。在涵盖多种前沿生成模型合成图像的综合性数据集上进行了广泛实验。结果表明，FIDAVL在假图像检测上达到平均95.42%的准确率和95.47%的F1分数；在图像溯源任务中，平均F1分数为92.64%，ROUGE-L得分为96.50%。相关代码将公开于https://github.com/Mamadou-Keita/FIDAVL。
原文摘要 · Abstract (English)
We introduce FIDAVL: Fake Image Detection and Attribution using a Vision-Language Model. FIDAVL is a novel and efficient mul-titask approach inspired by the synergies between vision and language processing. Leveraging the benefits of zero-shot learning, FIDAVL exploits the complementarity between vision and language along with soft prompt-tuning strategy to detect fake images and accurately attribute them to their originating source models. We conducted extensive experiments on a comprehensive dataset comprising synthetic images generated by various state-of-the-art models. Our results demonstrate that FIDAVL achieves an encouraging average detection accuracy of 95.42% and F1-score of 95.47% while also obtaining noteworthy performance metrics, with an average F1-score of 92.64% and ROUGE-L score of 96.50% for attributing synthetic images to their respective source generation models. The source code of this work will be publicly released at https://github.com/Mamadou-Keita/FIDAVL.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。