arXiv:2603.21054cs.LGcs.AI2026-03

区分视觉内容操纵的有害与无害意图,能显著提升多媒体虚假信息检测效果。

Harmful Visual Content Manipulation Matters in Misinformation Detection Under Multimedia Scenarios

  • 引入双重特征:视觉篡改痕迹与篡改意图类型
  • 在四个数据集上均显著优于现有方法
  • 适用于社交媒体虚假信息识别场景

当前社交媒体中虚假信息广泛传播,对社会造成严重负面影响。为此,多模态虚假信息检测(MMD)成为研究热点,主流方法关注跨模态语义一致性,却常忽略视觉内容中的关键线索。已有研究指出,社交媒体图文中的视觉篡改特征是重要判别信号。本文进一步提出:篡改背后的意图(有害或无害)同样关键。因此,我们提出同时捕捉两类特征:篡改特征(是否被篡改)与意图特征(篡改动机)。由于缺乏标注的篡改与意图标签,我们采用弱监督策略,结合图像篡改检测数据集,将分类任务建模为正/未标记学习。基于此,提出新型MMD方法HAVC-M4D。在四个主流MMD数据集上的实验表明,该方法能持续显著提升现有模型性能。

原文摘要 · Abstract (English)

Nowadays, the widespread dissemination of misinformation across numerous social media platforms has led to severe negative effects on society. To address this challenge, the automatic detection of misinformation, particularly under multimedia scenarios, has gained significant attention from both academic and industrial communities, leading to the emergence of a research task known as Multimodal Misinformation Detection (MMD). Typically, current MMD approaches focus on capturing the semantic relationships and inconsistency between various modalities but often overlook certain critical indicators within multimodal content. Recent research has shown that manipulated features within visual content in social media articles serve as valuable clues for MMD. Meanwhile, we argue that the potential intentions behind the manipulation, e.g., harmful and harmless, also matter in MMD. Therefore, in this study, we aim to identify such multimodal misinformation by capturing two types of features: manipulation features, which represent if visual content has been manipulated, and intention features, which assess the nature of these manipulations, distinguishing between harmful and harmless intentions. Unfortunately, the manipulation and intention labels that supervise these features to be discriminative are unknown. To address this, we introduce two weakly supervised indicators as substitutes by incorporating supplementary datasets focused on image manipulation detection and framing two different classification tasks as positive and unlabeled learning issues. With this framework, we introduce an innovative MMD approach, titled Harmful Visual Content Manipulation Matters in MMD (HAVC-M4 D). Comprehensive experiments conducted on four prevalent MMD datasets indicate that HAVC-M4 D significantly and consistently enhances the performance of existing MMD methods.

虚假信息检测多模态视觉篡改弱监督

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。