首个支持图文联合造假检测的基准，助力可信信息识别。
VLDBench Evaluating Multimodal Disinformation with Regulatory Alignment
- 构建包含6.2万组图文对的多模态假信息数据集
- 图文结合模型比纯文本模型准确率提升5至35个百分点
- 符合主流AI治理框架，适合研究者与政策制定者使用
识别融合伪造文本与图像的误导性内容日益困难，因生成式AI使合成内容易于生产与传播。现有AI安全评测多聚焦单模态虚假信息（即无欺骗意图的内容），而有意制造的多模态假信息（如模仿可信新闻的宣传或阴谋论）仍缺乏系统评估。本文提出视觉-语言假信息检测基准VLDBench，是首个大规模资源，支持单模态（仅文本）与多模态（文本+图像）假信息检测。该数据集包含约62,000个标注图文对，覆盖13类主题，源自58家新闻机构。通过半自动化流程结合专家评审，22位领域专家投入超500小时，达成较高标注一致性。对先进大语言模型（LLMs）与视觉-语言模型（VLMs）的评估显示，引入视觉线索可使检测准确率较纯文本模型提升5至35个百分点。项目提供数据与代码，支持评估、微调与鲁棒性测试。开发过程契合人工智能治理框架（如MIT AI风险库），为多模态媒体中可信假信息检测提供坚实基础。
原文摘要 · Abstract (English)
Detecting disinformation that blends manipulated text and images has become increasingly challenging, as AI tools make synthetic content easy to generate and disseminate. While most existing AI safety benchmarks focus on single modality misinformation (i.e., false content shared without intent to deceive), intentional multimodal disinformation, such as propaganda or conspiracy theories that imitate credible news, remains largely unaddressed. We introduce the Vision-Language Disinformation Detection Benchmark (VLDBench), the first large-scale resource supporting both unimodal (text-only) and multimodal (text + image) disinformation detection. VLDBench comprises approximately 62,000 labeled text-image pairs across 13 categories, curated from 58 news outlets. Using a semi-automated pipeline followed by expert review, 22 domain experts invested over 500 hours to produce high-quality annotations with substantial inter-annotator agreement. Evaluations of state-of-the-art Large Language Models (LLMs) and Vision-Language Models (VLMs) on VLDBench show that incorporating visual cues improves detection accuracy by 5 to 35 percentage points over text-only models. VLDBench provides data and code for evaluation, fine-tuning, and robustness testing to support disinformation analysis. Developed in alignment with AI governance frameworks (e.g., the MIT AI Risk Repository), VLDBench offers a principled foundation for advancing trustworthy disinformation detection in multimodal media. Project: https://vectorinstitute.github.io/VLDBench/ Dataset: https://huggingface.co/datasets/vector-institute/VLDBench Code: https://github.com/VectorInstitute/VLDBench
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。