用DINOv2定位生成式修复中的篡改区域，效果优于现有方法。
DinoLizer: Separating VAE and Diffusion Artifacts in Generative Inpainting Localization
- 将修复区外重建内容视为独立类别，引导模型关注语义变化区。
- 在14×14特征图上实现定位，平均交并比高出第二好模型20%。
- 轻量微调+滑动窗口设计，对双压缩图像鲁棒性强，适合检测工具开发。
我们提出DinoLizer，一种基于DINOv2的生成式修复篡改区域定位模型。通过将掩码外重建区域作为独立类别进行训练，使模型更聚焦于语义改变区域，显著优于传统方法。采用LoRA微调Transformer块的查询与值，并在主干网络后添加一层线性层，实现14×14分辨率下的篡改预测。由于DINOv2仅支持固定尺寸输入，采用滑动窗口聚合大图预测结果。实验证明，DinoLizer在自建数据集及SOTA修复数据集上均超越现有方法，且对双层JPEG压缩具有强鲁棒性。平均交并比（IoU）较次优模型提升20%。代码已公开：https://github.com/anonyme610/dinolizer。
原文摘要 · Abstract (English)
We introduce DinoLizer, a DINOv2-based localizer of manipulated areas in generative inpainting. The model is trained to focus on semantically altered regions by treating reconstructed areas outside the inpainted mask as a separate class, which yields significant improvements w.r.t. the conven- tional approach. We train the model with LORA on the Query and Value of the transformer blocks and simply add 1 linear layer on top of the backbone to predict manipulations on a 14 x 14 patch resolution. Because DINOv2 only accepts fixed- sized images, we use a sliding window approach to aggregate the predictions on larger images. Empirical results show that DinoLizer outperforms state-of-the-art methods on our proposed dataset and SOTA inpainting datasets. Furthermore, it is very robust to JPEG (double) compression. On average, DinoLizer achieves a 20% higher Intersection over Union score compared to the second best model. The code is publicly available here: https://github.com/anonyme610/dinolizer.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。