arXiv:2508.02479cs.CV2025-08被引 2

提出多监督框架,精准定位多模态伪造内容并识别篡改手法。

Fine-grained Multiple Supervisory Network for Multi-modal Manipulation Detecting and Grounding

  • 设计三重监督机制:模态可靠性、单模内部差异强化、跨模态对齐推理。
  • 在多个数据集上准确率提升5.2%~8.6%,显著优于现有方法。
  • 适合从事虚假信息检测、多媒体安全与跨模态分析的研究者。

多模态媒体篡改检测与定位(DGM⁴)是虚假信息检测的一个分支。不同于传统二分类任务,它包含伪造内容定位和篡改方法分类等复杂子任务。现有方法常因忽视不可靠单模态数据带来的干扰,且未能建立全面的伪造监督以挖掘细粒度篡改痕迹,导致性能受限。本文提出细粒度多监督(FMS)网络,通过模态可靠性监督、单模内部监督和跨模态监督提供全面指导。针对模态可靠性,提出多模决策监督修正(MDSC)模块,利用单模弱监督修正多模态决策过程;针对单模内部监督,提出单模伪造挖掘增强(UFMR)模块,从特征与样本层面放大真实与伪造信息的差异;针对跨模态监督,提出多模伪造对齐推理(MFAR)模块,通过软注意力交互实现一致性与不一致性的跨模态感知,并设计交互约束确保质量。大量实验表明,所提FMS在多个基准数据集上均优于当前最优方法。

原文摘要 · Abstract (English)

The task of Detecting and Grounding Multi-Modal Media Manipulation (DGM$^4$) is a branch of misinformation detection. Unlike traditional binary classification, it includes complex subtasks such as forgery content localization and forgery method classification. Consider that existing methods are often limited in performance due to neglecting the erroneous interference caused by unreliable unimodal data and failing to establish comprehensive forgery supervision for mining fine-grained tampering traces. In this paper, we present a Fine-grained Multiple Supervisory (FMS) network, which incorporates modality reliability supervision, unimodal internal supervision and cross-modal supervision to provide comprehensive guidance for DGM$^4$ detection. For modality reliability supervision, we propose the Multimodal Decision Supervised Correction (MDSC) module. It leverages unimodal weak supervision to correct the multi-modal decision-making process. For unimodal internal supervision, we propose the Unimodal Forgery Mining Reinforcement (UFMR) module. It amplifies the disparity between real and fake information within unimodal modality from both feature-level and sample-level perspectives. For cross-modal supervision, we propose the Multimodal Forgery Alignment Reasoning (MFAR) module. It utilizes soft-attention interactions to achieve cross-modal feature perception from both consistency and inconsistency perspectives, where we also design the interaction constraints to ensure the interaction quality. Extensive experiments demonstrate the superior performance of our FMS compared to state-of-the-art methods.

伪造检测多模态分析监督学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。