用冻结的DINO模型无需训练就能定位图像编辑区域
Frozen DINO Localizes Image Edits Without a Localizer
- 通过全局哈尔噪声扰动捕捉特征块漂移，生成定位图
- 在CocoGlide数据集上达0.903的补丁AUROC，接近有监督方法
- 适用于无生成器场景，适合图像取证与安全研究
局部图像编辑可改变照片含义而保留大部分内容真实，因此需要识别编辑位置。我们发现冻结DINO编码器的块级扰动响应本身即可作为定位图。无需训练的TRAIL方法通过一次全局哈尔扰动，测量对应块特征之间的余弦漂移来实现定位。在80张源不重叠的CocoGlide测试图像上，TRAIL的补丁AUROC达0.903，接近有掩码监督的Detective SAM（0.912）；固定阈值下Dice为0.619，而使用最优阈值可提升至0.790。在泊松图像插值场景中，TRAIL仍达0.855 AUROC，证明该信号不依赖生成器。十六个DINO编码器中，最佳块位于归一化深度0.80–0.94之间。全局上下文至关重要：全局扰动下AUROC为0.903，局部画布内扰动降至0.857，独立编码的裁剪图像仅为0.735。冻结的DINO块特征蕴含强晚层定位信号，其可见性取决于扰动方式和上下文完整性。
原文摘要 · Abstract (English)
Localized image edits can change a photograph's meaning while leaving most of it authentic, so forensic analysis must identify where an edit occurred. We show that patch-level perturbation responses from frozen DINO encoders are themselves localization maps. Training-free Localization of AI-image Edits from patch-token Drift (TRAIL) applies one global Haar perturbation and maps cosine drift between corresponding patch tokens. On 80 source-disjoint CocoGlide test images, TRAIL reaches .903 patch AUROC versus .912 for the mask-supervised Detective SAM; fixed-threshold Dice is .619 versus .709, while an oracle threshold raises TRAIL to .790. Transferred unchanged to Poisson image interpolation, TRAIL reaches .855 AUROC versus .864, showing that the cue persists without a generator. Across sixteen DINO encoders, the best block lies at normalized depth .80-.94. Global context matters: AUROC falls from .903 globally to .857 for local-in-canvas perturbations and .735 for independently encoded crops. Frozen DINO patch tokens therefore contain a strong late-layer localization signal whose visibility depends on the perturbation and preserved context. Code: https://github.com/VishalJ99/trail-image-edit-localization.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。