用特征解量化提升图像修复细节质量，效果显著且开销极小。
Improving Detail in Pluralistic Image Inpainting with Feature Dequantization
- 提出特征解量化模块FDM，补偿量化导致的信息损失。
- 在PUT模型基础上提升细节质量，显著改善生成图像清晰度。
- 训练高效，对现有流程几乎无额外负担，适合实际部署。
多解图像修复（PII）能为图像缺失区域生成多个合理补全方案，广泛应用于图像编辑与物体移除。近期基于VQGAN的方法显著提升了生成图像的结构完整性，但当前最先进的VQGAN方法PUT存在关键问题：特征量化导致输出图像细节质量下降。特征量化限制了潜在空间并造成信息丢失，严重影响图像修复所需的细节表现力。为此，我们提出专用于恢复细节质量的特征解量化模块（FDM），通过补偿信息损失来提升生成效果。同时设计了一种高效的FDM训练方法，大幅降低训练成本。实验证明，该方法在几乎不增加训练和推理开销的前提下，显著提升了生成图像的细节质量。
原文摘要 · Abstract (English)
Pluralistic Image Inpainting (PII) offers multiple plausible solutions for restoring missing parts of images and has been successfully applied to various applications including image editing and object removal. Recently, VQGAN-based methods have been proposed and have shown that they significantly improve the structural integrity in the generated images. Nevertheless, the state-of-the-art VQGAN-based model PUT faces a critical challenge: degradation of detail quality in output images due to feature quantization. Feature quantization restricts the latent space and causes information loss, which negatively affects the detail quality essential for image inpainting. To tackle the problem, we propose the FDM (Feature Dequantization Module) specifically designed to restore the detail quality of images by compensating for the information loss. Furthermore, we develop an efficient training method for FDM which drastically reduces training costs. We empirically demonstrate that our method significantly enhances the detail quality of the generated images with negligible training and inference overheads.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。