构建首个多模态伪造内容数据集,验证AI能否追溯并解释生成来源。
Could AI Trace and Explain the Origins of AI-Generated Images and Text?
- 构建含28万样本的AI伪造内容数据集,覆盖图文生成与恶意用途。
- 发现模型生成内容的原始训练意图影响其可溯源性。
- GPT-4o能一致解释来源但不够具体,尤其对自家模型如DALL-E
AI生成内容在现实世界日益普及,引发严重的伦理与社会问题。例如,攻击者可能利用大模型生成违反伦理或法律的图像,而论文审稿人可能滥用大语言模型生成无真实思考的评审意见。尽管已有研究探索检测和追溯生成内容来源,但缺乏系统且细粒度的对比分析。重要维度如图像与文本生成、完全与部分生成、通用与恶意使用场景仍待深入。此外,现有研究未解决一个关键问题:AI系统(如GPT-4o)能否解释为何某内容被归因于特定生成模型。为此,我们提出AI-FAKER,一个包含超过28万样本的综合性多模态数据集,涵盖多种大语言模型(LLMs)与大模型(LMMs),覆盖图像与文本的通用及恶意使用场景。实验揭示两个核心发现:(i) AI作者溯源不仅依赖生成输出,还受原始模型训练意图影响;(ii) GPT-4o在分析来自OpenAI自身模型(如DALL-E和GPT-4o)的内容时,虽提供高度一致的解释,但准确性较低,缺乏细节。
原文摘要 · Abstract (English)
AI-generated content is becoming increasingly prevalent in the real world, leading to serious ethical and societal concerns. For instance, adversaries might exploit large multimodal models (LMMs) to create images that violate ethical or legal standards, while paper reviewers may misuse large language models (LLMs) to generate reviews without genuine intellectual effort. While prior work has explored detecting AI-generated images and texts, and occasionally tracing their source models, there is a lack of a systematic and fine-grained comparative study. Important dimensions--such as AI-generated images vs. text, fully vs. partially AI-generated images, and general vs. malicious use cases--remain underexplored. Furthermore, whether AI systems like GPT-4o can explain why certain forged content is attributed to specific generative models is still an open question, with no existing benchmark addressing this. To fill this gap, we introduce AI-FAKER, a comprehensive multimodal dataset with over 280,000 samples spanning multiple LLMs and LMMs, covering both general and malicious use cases for AI-generated images and texts. Our experiments reveal two key findings: (i) AI authorship detection depends not only on the generated output but also on the model's original training intent; and (ii) GPT-4o provides highly consistent but less specific explanations when analyzing content produced by OpenAI's own models, such as DALL-E and GPT-4o itself.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。