arXiv:2508.07216cs.CV2025-08被引 2

利用认知启发的图文协同机制,精准定位图像篡改区域。

Bridging Semantic Logic Gaps: A Cognition Inspired Multimodal Boundary Preserving Network for Image Manipulation Localization

  • 引入大模型生成文本提示,补全视觉信息缺失的语义逻辑
  • 通过模糊度加权确保文本信息不因幻觉干扰而失真
  • 基于可逆网络保留篡改边界细节,适合高精度反欺诈场景

现有图像篡改定位(IML)模型主要依赖视觉线索,忽视内容特征间的语义逻辑关系。真实图像的内容语义通常符合人类认知规律,而图像篡改技术常破坏这种内在关联,留下可用于定位的语义线索。本文提出一种认知启发的多模态边界保持网络(CMB-Net)。CMB-Net利用大语言模型(LLMs)分析图像中的篡改区域,并生成基于提示的文本信息,以弥补视觉信息中语义关系的缺失。为防止大模型幻觉导致的错误文本损害定位精度,提出图像-文本中心模糊度模块(ITCAM),通过量化图文特征间的模糊度对文本特征进行加权,确保文本信息的有益作用。同时设计图像-文本交互模块(ITIM),利用相关矩阵实现视觉与文本特征的细粒度对齐。最后,受可逆神经网络启发,提出还原边缘解码器(RED),通过双向生成输入与输出特征,在不损失信息的前提下保留篡改区域的边界细节。大量实验表明,CMB-Net优于多数现有IML模型。代码已开源:https://github.com/vpsg-research/CMB-Net。

原文摘要 · Abstract (English)

The existing image manipulation localization (IML) models mainly relies on visual cues, but ignores the semantic logical relationships between content features. In fact, the content semantics conveyed by real images often conform to human cognitive laws. However, image manipulation technology usually destroys the internal relationship between content features, thus leaving semantic clues for IML. In this paper, we propose a cognition inspired multimodal boundary preserving network (CMB-Net). Specifically, CMB-Net utilizes large language models (LLMs) to analyze manipulated regions within images and generate prompt-based textual information to compensate for the lack of semantic relationships in the visual information. Considering that the erroneous texts induced by hallucination from LLMs will damage the accuracy of IML, we propose an image-text central ambiguity module (ITCAM). It assigns weights to the text features by quantifying the ambiguity between text and image features, thereby ensuring the beneficial impact of textual information. We also propose an image-text interaction module (ITIM) that aligns visual and text features using a correlation matrix for fine-grained interaction. Finally, inspired by invertible neural networks, we propose a restoration edge decoder (RED) that mutually generates input and output features to preserve boundary information in manipulated regions without loss. Extensive experiments show that CMB-Net outperforms most existing IML models. Our code is available on https://github.com/vpsg-research/CMB-Net.

图像篡改多模态融合边界保持大模型应用

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。