通过比对真实证据检测图文伪造,实现无需重训的跨域适应。
REVEAL: Reference-Grounded Reasoning for Multimodal Manipulation Detection

- 以参考证据对比方式判断图文真伪,模拟人类推理逻辑。
- 在17万真实图文对上训练,检测准确率显著超越现有方法。
- 更新参考库即可应对新领域,适合反虚假信息系统部署。
多模态篡改检测旨在同时识别伪造的图文对并定位篡改区域,但现有方法通常依赖记忆孤立痕迹,在难以察觉的篡改或领域迁移下表现不佳。受人类对比推理启发,我们提出将该任务重构为基于参考的验证问题:通过对比查询样本与检索到的真实证据来评估真实性。为此,我们构建了一个包含17万条真实新闻图文对的大规模参考库,涵盖4万余名公众人物。技术上,REVEAL采用差异感知融合机制,捕捉查询与参考证据间的细微差异;并引入任务解耦的Mixture-of-Experts(MoE)架构,协同完成实例级检测与细粒度定位,有效缓解两类异构目标间的优化冲突。大量实验表明,REVEAL显著优于当前最优方法,并可通过仅更新参考库实现无训练域自适应,为应对持续演化的虚假信息提供鲁棒且实用的解决方案。代码已公开于https://anonymous.4open.science/r/REVEAL-Reference-A006。
原文摘要 · Abstract (English)
Multimodal manipulation detection aims to simultaneously identify forged image--text pairs and localize tampered regions, yet existing methods typically rely on memorizing isolated artifacts and struggle with imperceptible manipulation traces or domain shifts. Inspired by human comparative reasoning, we reformulate this task as a reference-grounded verification problem, where authenticity is assessed by comparing a query against retrieved authentic evidence. We propose REVEAL Reference-Enabled Verification for Evidence Analysis and Localization), a framework explicitly designed for this comparative paradigm. To support this paradigm, we construct a large-scale reference library comprising 170K authentic news image--text pairs featuring over 40K public figures. Technically, REVEAL employs a difference-aware fusion mechanism to capture fine-grained discrepancies between the query and retrieved evidence. Furthermore, we introduce a task-decoupled Mixture-of-Experts (MoE) architecture to jointly execute instance-level detection and fine-grained grounding, effectively mitigating optimization conflicts between these heterogeneous objectives. Extensive experiments demonstrate that REVEAL significantly outperforms state-of-the-art methods, and notably enables \emph{training-free domain adaptation} by simply updating the reference library, offering a robust and practical solution for detecting evolving misinformation. Code is available at https://anonymous.4open.science/r/REVEAL-Reference-A006.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。