测试了14种AI图像检测工具在含文字伪造图上的表现，发现准确率普遍低于80%。
TextFake: Benchmarking AI-Generated Image Detection on Text-Rich Images
- 构建2万张多语言文本图像伪造数据集，模拟真实谣言场景。
- 多数检测器在含密集文字的图像上准确率下降超60%。
- 揭示三大失效机制，适合做AI内容安全研究者参考。
近期的AI生成图像（AIGI）检测器在自然图像基准上表现良好，但在包含文字的伪造图像（如虚假截图、文档、新闻页等）上的表现尚未被验证。本文提出TextFake，一个包含20,000张图像的基准数据集，覆盖28种语言、4个主题类别和2种场景模态。通过四阶段流程，对真实图像进行三维度标注，并采用分布对齐的结构化提示生成伪造图像，排除协变量捷径。对14种专用检测器和3个前沿视觉语言模型API进行零样本评估，发现系统性差距：无一方法准确率超过80%，部分较自然图像基准下降超60%。诊断分析揭示三种失效模式：文本密度诅咒（密集文字淹没低层特征）、渲染保真度伪装（更强文本渲染掩盖生成痕迹）、阈值崩溃（常规扰动使性能退至随机水平）。
原文摘要 · Abstract (English)
Recent AI-generated image (AIGI) detectors perform well on natural-image benchmarks, but their behavior on text-rich forgeries, such as fabricated screenshots, documents, and news pages prevalent in misinformation, remains untested. We introduce TextFake, a 20,000-image benchmark for text-rich AIGI detection spanning 28 languages, 4 topic categories, and 2 scene modalities. Fake images are synthesized via a four-stage pipeline that annotates real images along three controlled dimensions and generates counterparts through distribution-aligned structured prompting, ruling out covariate shortcuts. Zero-shot evaluation of 14 specialized detectors and 3 frontier VLM APIs reveals a large systematic gap: no method exceeds 80% accuracy, with some dropping over 60% from natural-image benchmarks. Diagnostic evaluations identify three failure modes: the Text Density Curse, where dense glyphs overwhelm low-level detectors; Cloaking via Rendering Fidelity, where stronger text rendering suppresses enerative artifacts; and Threshold Collapse, where routine perturbations drive detectors toward chance-level performance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。