用粗略标注实现精准图像篡改定位,兼顾效率与准确性
From Passive Perception to Active Memory: A Weakly Supervised Image Manipulation Localization Framework Driven by Coarse-Grained Annotations

- 用粗粒度区域标注替代像素级标注,降低人工成本
- 轻量学生模型通过知识蒸馏,达到接近全监督的定位精度
- 模拟人脑记忆机制,动态融合长期记忆与实时感知,提升鲁棒性
图像篡改定位(IML)面临标注成本与精确定位精度之间的根本矛盾。现有全监督方法依赖密集的像素级掩码标注,难以扩展至大规模数据集或真实场景。而多数弱监督方法仅使用图像级标签,虽大幅减少标注负担,但空间定位精度不足。为此,本文提出BoxPromptIML框架,采用粗粒度区域标注策略,在较低成本下生成相对准确的篡改掩码。为提升模型效率与部署可行性,设计了一个轻量级学生模型,通过知识蒸馏从固定教师模型(基于Segment Anything Model, SAM)中学习细粒度定位能力。此外,受人类潜意识记忆机制启发,特征融合模块采用双引导策略,主动将回忆的原型模式与输入图像的实时观测线索进行上下文对齐。该动态知识回溯机制突破了被动特征提取局限,使长时记忆能适应当前图像语境,显著提升定位准确率与鲁棒性。在分布内与分布外数据集上的大量实验表明,BoxPromptIML性能优于或媲美全监督模型,同时保持强泛化能力、低标注成本与高效部署特性。
原文摘要 · Abstract (English)
Image manipulation localization (IML) faces a fundamental trade-off between minimizing annotation cost and achieving fine-grained localization accuracy. Existing fully-supervised IML methods depend heavily on dense pixel-level mask annotations, which limits scalability to large datasets or real-world deployment.In contrast, the majority of existing weakly-supervised IML approaches are based on image-level labels, which greatly reduce annotation effort but typically lack precise spatial localization. To address this dilemma, we propose BoxPromptIML, a novel weakly-supervised IML framework that effectively balances annotation cost and localization performance. Specifically, we propose a coarse region annotation strategy, which can generate relatively accurate manipulation masks at lower cost. To improve model efficiency and facilitate deployment, we further design an efficient lightweight student model, which learns to perform fine-grained localization through knowledge distillation from a fixed teacher model based on the Segment Anything Model (SAM). Moreover, inspired by the human subconscious memory mechanism, our feature fusion module employs a dual-guidance strategy that actively contextualizes recalled prototypical patterns with real-time observational cues derived from the input. Instead of passive feature extraction, this strategy enables a dynamic process of knowledge recollection, where long-term memory is adapted to the specific context of the current image, significantly enhancing localization accuracy and robustness. Extensive experiments across both in-distribution and out-of-distribution datasets show that BoxPromptIML outperforms or rivals fully-supervised models, while maintaining strong generalization, low annotation cost, and efficient deployment characteristics.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。