动态调整检测提示,让模型更准识别未知生成图像。
Towards Generalizable AI-Generated Image Detection via Image-Adaptive Prompt Learning
- 测试时根据每张图自适应调整提示词,提升泛化能力。
- 在UniversalFakeDetect和GenImage数据集上准确率分别达95.61%和96.7%。
- 适合需要应对新型生成图像的检测场景,如内容审核与溯源。
在AI生成图像检测中,当前先进方法通常通过部分参数微调预训练基础模型,但这些方法难以泛化到未见过的生成器所制造的伪造图像,因其仅捕捉训练数据中的有限模式,无法反映新生成器的演变特征。为此,我们提出图像自适应提示学习(IAPL),一种新范式:在测试阶段根据每张待测图像动态调整输入编码器的提示词,而非训练后固定提示。该设计显著增强对多样化伪造图像的鲁棒性与适应性。动态提示通过轻量级可学习缩放因子,将条件信息与测试时自适应标记相结合。条件信息由基于CNN的特征提取器生成,建模特定伪造特征与通用条件。测试时自适应标记通过强制多视图预测一致性,在单样本推理过程中优化,确保参数与当前图像匹配。最终决策采用预测置信度最高的输入。大量实验表明,IAPL达到当前最优性能,在通用伪造检测(UniversalFakeDetect)和GenImage数据集上平均准确率分别为95.61%和96.7%。代码与权重将发布于https://github.com/liyih/IAPL。
原文摘要 · Abstract (English)
In AI-generated image detection, current cutting-edge methods typically adapt pre-trained foundation models through partial-parameter fine-tuning. However, these approaches often struggle to generalize to forgeries from unseen generators, as the fine-tuned models capture only limited patterns from training data and fail to reflect the evolving traits of new ones. To overcome this limitation, we propose Image-Adaptive Prompt Learning (IAPL), a novel paradigm that dynamically adjusts the prompts fed into the encoder according to each testing image, rather than fixing them after training. This design significantly enhances robustness and adaptability to diverse forged images. The dynamic prompts integrate conditional information with test-time adaptive tokens through a lightweight learnable scaling factor. The conditional information is produced by a Conditional Information Learner, which leverages CNN-based feature extractors to model both forgery-specific and general conditions. The test-time adaptive tokens are optimized during inference on a single sample by enforcing prediction consistency across multiple views, ensuring that the parameters align with the current image. For the final decision, the optimal input with the highest prediction confidence is selected. Extensive experiments show that IAPL achieves state-of-the-art performance, with mean accuracies of 95.61% and 96.7% on the widely used UniversalFakeDetect and GenImage datasets, respectively. Codes and weights will be released on https://github.com/liyih/IAPL.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。