通过可调提示控制图像修复，实现对象生成与背景延续的精准混合。
ControlFill: Spatially Adjustable Image Inpainting from Prompt Learning
- 用两个独立提示分别控制对象生成和背景延续
- 支持按像素调节修复强度，实现空间可调修复
- 无需复杂文本编码器，推理高效适合交互应用
本文提出一种名为ControlFill的图像修复框架，通过训练两个不同提示：一个用于在指定掩码区域生成合理物体（创建），另一个用于通过延伸背景填充区域（移除）。推理阶段，这些学习到的嵌入引导扩散网络运行，无需依赖重型文本编码器。通过调整两个提示的相对重要性并结合无分类器指导，用户可控制修复中移除或创建的强度。此外，提出一种方法，通过为每个像素分配不同尺度，实现引导强度的空间变化，从而实现局部精细化控制。
原文摘要 · Abstract (English)
In this report, I present an inpainting framework named \textit{ControlFill}, which involves training two distinct prompts: one for generating plausible objects within a designated mask (\textit{creation}) and another for filling the region by extending the background (\textit{removal}). During the inference stage, these learned embeddings guide a diffusion network that operates without requiring heavy text encoders. By adjusting the relative significance of the two prompts and employing classifier-free guidance, users can control the intensity of removal or creation. Furthermore, I introduce a method to spatially vary the intensity of guidance by assigning different scales to individual pixels.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。