让SAM自动识别图像篡改，无需人工提示
IMDPrompter: Adapting SAM to Image Manipulation Detection by Cross-View Automated Prompt Learning
- 设计自动提示机制，实现无手动干预的检测
- 跨视角学习提升在多个数据集上的泛化能力
- 适合图像安全、AI审计等领域的研究人员
基于SA-1B大规模训练数据，通用分割模型SAM展现出卓越的泛化与零样本能力，广泛应用于医学和遥感图像分割。然而其在图像篡改检测领域的表现尚未深入探索。现有应用面临两大挑战：依赖人工提示，以及单视角信息难以支持跨数据集泛化。为此，我们提出基于SAM的跨视角自动提示学习框架IMDPrompter。通过自动化提示设计，该方法摆脱对人工引导的依赖，实现全自动检测与定位。同时引入跨视角特征感知、最优提示选择与跨视角提示一致性组件，促进跨视角感知学习，引导SAM生成精准掩码。在五个数据集（CASIA、Columbia、Coverage、IMD2020、NIST16）上的大量实验验证了该方法的有效性。
原文摘要 · Abstract (English)
Using extensive training data from SA-1B, the Segment Anything Model (SAM) has demonstrated exceptional generalization and zero-shot capabilities, attracting widespread attention in areas such as medical image segmentation and remote sensing image segmentation. However, its performance in the field of image manipulation detection remains largely unexplored and unconfirmed. There are two main challenges in applying SAM to image manipulation detection: a) reliance on manual prompts, and b) the difficulty of single-view information in supporting cross-dataset generalization. To address these challenges, we develops a cross-view prompt learning paradigm called IMDPrompter based on SAM. Benefiting from the design of automated prompts, IMDPrompter no longer relies on manual guidance, enabling automated detection and localization. Additionally, we propose components such as Cross-view Feature Perception, Optimal Prompt Selection, and Cross-View Prompt Consistency, which facilitate cross-view perceptual learning and guide SAM to generate accurate masks. Extensive experimental results from five datasets (CASIA, Columbia, Coverage, IMD2020, and NIST16) validate the effectiveness of our proposed method.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。