arXiv:2508.01338cs.CVcs.AI2025-08AAAI被引 5

用图文协同推理,仅靠图像标签实现精准伪造定位。

Weakly-Supervised Image Forgery Localization via Vision-Language Collaborative Reasoning Framework

  • 引入预训练图文模型提取语义先验,弥补弱监督不足。
  • 通过双头预测与对比块一致性模块,实现像素级定位。
  • 适合需要低标注成本的图像真实性检测场景。

图像伪造定位旨在精确识别图像中被篡改的区域,但传统方法依赖昂贵的像素级标注。为减轻标注负担,弱监督图像伪造定位(WSIFL)应运而生,然而现有方法因主要依赖图像内部一致性线索,缺乏外部语义引导,定位性能受限。本文提出ViLaCo:一种视觉-语言协同推理框架,从预训练视觉-语言模型(VLMs)中提炼辅助语义监督,仅使用图像级标签即可实现像素级定位。具体地,ViLaCo首先通过视觉-语言特征建模网络,联合提取文本与视觉先验;随后,自适应视觉-语言推理网络通过双向交互对齐语义与视觉特征,生成语义一致表示;再经双预测头处理,粗粒度头完成图像级分类,细粒度头生成像素级定位掩码,弥合弱监督与细粒度定位的差距。此外,引入对比块一致性模块,聚类篡改特征并分离真实区域,提升伪造判别可靠性。在多个公开数据集上的大量实验表明,ViLaCo显著优于现有WSIFL方法,在检测与定位精度上均达到当前最优水平。

原文摘要 · Abstract (English)

Image forgery localization aims to precisely identify tampered regions within images, but it commonly depends on costly pixel-level annotations. To alleviate this annotation burden, weakly supervised image forgery localization (WSIFL) has emerged, yet existing methods still achieve limited localization performance as they mainly exploit intra-image consistency clues and lack external semantic guidance to compensate for weak supervision. In this paper, we propose ViLaCo, a vision-language collaborative reasoning framework that introduces auxiliary semantic supervision distilled from pre-trained vision-language models (VLMs), enabling accurate pixel-level localization using only image-level labels. Specifically, ViLaCo first incorporates semantic knowledge through a vision-language feature modeling network, which jointly extracts textual and visual priors using pre-trained VLMs. Next, an adaptive vision-language reasoning network aligns textual semantics and visual features through mutual interactions, producing semantically aligned representations. Subsequently, these representations are passed into dual prediction heads, where the coarse head performs image-level classification and the fine head generates pixel-level localization masks, thereby bridging the gap between weak supervision and fine-grained localization. Moreover, a contrastive patch consistency module is introduced to cluster tampered features while separating authentic ones, facilitating more reliable forgery discrimination. Extensive experiments on multiple public datasets demonstrate that ViLaCo substantially outperforms existing WSIFL methods, achieving state-of-the-art performance in both detection and localization accuracy.

伪造定位弱监督图文协同VLM

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。