用BERT和CLIP检测AI生成图像并识别模型来源，效果优于多数现有方法。
NAU-QMUL: Utilizing BERT and CLIP for Multi-modal AI-Generated Image Detection
- 融合BERT与CLIP的跨模态特征提取，支持图文联合分析。
- 在竞赛中任务A、B的F1得分分别达83.16%和48.88%，排名第五。
- 采用伪标签增强数据，提升模型对未知生成模型的泛化能力。
为实现AI生成图像的检测及具体生成模型的识别，本文提出一种多模态多任务模型。该模型分别利用预训练BERT和CLIP视觉编码器提取文本与图像特征，并通过定制化的多任务损失函数实现跨模态特征融合。同时，采用基于伪标签的数据增强策略，引入高置信度样本扩充训练集。在`CT2: AI-Generated Image Detection'竞赛中，该模型在任务A和任务B上分别取得83.16%和48.88%的F1分数，位列第五。结果表明，所提架构在真实场景下具备较强的AI生成内容检测能力。代码已开源：https://github.com/xxxxxxxxy/AIGeneratedImageDetection。
原文摘要 · Abstract (English)
With the aim of detecting AI-generated images and identifying the specific models responsible for their generation, we propose a multi-modal multi-task model. The model leverages pre-trained BERT and CLIP Vision encoders for text and image feature extraction, respectively, and employs cross-modal feature fusion with a tailored multi-task loss function. Additionally, a pseudo-labeling-based data augmentation strategy was utilized to expand the training dataset with high-confidence samples. The model achieved fifth place in both Tasks A and B of the `CT2: AI-Generated Image Detection' competition, with F1 scores of 83.16\% and 48.88\%, respectively. These findings highlight the effectiveness of the proposed architecture and its potential for advancing AI-generated content detection in real-world scenarios. The source code for our method is published on https://github.com/xxxxxxxxy/AIGeneratedImageDetection.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。