零样本生成误导性图文内容的上下文警告，助力自动辟谣。
Zero-Shot Warning Generation for Misinformative Multimodal Content
- 通过跨模态一致性检测识别图文错配的虚假内容
- 仅用三分之一参数量实现媲美主流模型的性能
- 零样本生成带上下文的辟谣警告，适合内容审核场景
虚假信息泛滥对社会构成严重威胁。其中，将真实图像与虚假文本组合而成的“脱离语境”式误导内容尤为隐蔽且易误导公众。现有检测方法多聚焦图像与文本的一致性评估，但普遍缺乏充分解释，难以有效辟谣。本文提出一种基于跨模态一致性检查的误导内容检测模型，训练成本极低。同时设计轻量化模型，参数量仅为同类模型的三分之一，仍保持优异性能。此外，引入双用途零样本学习任务，实现上下文相关警告的自动生成，支持自动化辟谣并提升用户理解。定性分析与人工评估显示该方法在生成警告方面具备潜力，但也存在局限性。
原文摘要 · Abstract (English)
The widespread prevalence of misinformation poses significant societal concerns. Out-of-context misinformation, where authentic images are paired with false text, is particularly deceptive and easily misleads audiences. Most existing detection methods primarily evaluate image-text consistency but often lack sufficient explanations, which are essential for effectively debunking misinformation. We present a model that detects multimodal misinformation through cross-modality consistency checks, requiring minimal training time. Additionally, we propose a lightweight model that achieves competitive performance using only one-third of the parameters. We also introduce a dual-purpose zero-shot learning task for generating contextualized warnings, enabling automated debunking and enhancing user comprehension. Qualitative and human evaluations of the generated warnings highlight both the potential and limitations of our approach.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。