解决图像修复中乱入物体和色彩不一致问题，提升修复真实感。
Aligned Stable Inpainting: Mitigating Unwanted Object Insertion and Preserving Color Consistency
- 用重建先验引导生成模型，抑制幻觉物体。
- 设计专用VAE解码器，实现局部色彩协调。
- 适用于大范围遮挡修复，适合图像编辑与修复场景。
生成式图像修复能在大而不规则的掩码下生成逼真结果，但现有方法仍存在两个常见问题：(1) 乱入物体：生成与上下文不符的幻觉内容；(2) 色彩不一致：导致纹理模糊的明显色偏。本文分析了问题成因，提出ASUKA框架——一种用于预训练修复模型的后处理方法。为减少乱入物体，采用基于重建的先验引导生成模型，在抑制幻觉的同时保持生成灵活性。为解决色彩不一致，设计专用VAE解码器，将隐空间到图像的解码建模为局部调和任务。在基于U-Net和DiT的修复模型上轻量级实现ASUKA。在Places2和自建基准MISATO上的实验表明，ASUKA有效抑制物体幻觉并提升色彩一致性，优于现有的扩散模型与修正流方法。数据集、模型与代码将开源。
原文摘要 · Abstract (English)
Generative image inpainting can produce realistic results even with large, irregular masks, but existing methods still suffer from two common problems: (1) Unwanted object insertion: hallucinate artifacts that do not match the surrounding context. (2) Color inconsistency: noticeable color shifts that lead to smeared textures. We analyze the causes of these issues and propose Aligned Stable inpainting with UnKnown Areas prior (ASUKA), a post-hoc framework for pre-trained inpainting models. To reduce unwanted object insertion, we use reconstruction-based priors to guide the generative model, suppressing hallucinated objects while preserving generative flexibility. To address color inconsistency, we design a specialized VAE decoder that formulates latent-to-image decoding as a local harmonization task. We implement ASUKA on both U-Net-based and DiT-based inpainting models with lightweight modifications. Experiments on Places2 and MISATO, our proposed benchmark, show that ASUKA effectively suppresses object hallucination and improves color consistency, outperforming existing diffusion- and rectified flow-based inpainting methods. The dataset, models, and code will be released on GitHub.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。