用修复模型拆解图像分层，实现自由编辑前景与背景。
From Inpainting to Layer Decomposition: Repurposing Generative Inpainting Models for Image Layer Decomposition
- 复用扩散修复模型，轻量微调实现图像分层分解
- 在合成数据上训练，物体移除与遮挡恢复效果更优
- 适合需要精细图像编辑的创意设计人员
图像可视为前景物体与背景的分层组合，存在潜在遮挡。这种分层表示支持元素独立编辑，提升内容创作灵活性。尽管生成模型发展迅速，单图分层仍因方法与数据有限而困难。我们发现分层分解与修复/补全任务有强关联，提出通过轻量微调将基于扩散的修复模型用于分层分解。为更好保留潜空间细节，引入线性注意力复杂度的多模态上下文融合模块。模型仅在开源资产构建的合成数据集上训练，便在物体移除和遮挡恢复任务中表现卓越，为下游编辑与创意应用开辟新可能。
原文摘要 · Abstract (English)
Images can be viewed as layered compositions, foreground objects over background, with potential occlusions. This layered representation enables independent editing of elements, offering greater flexibility for content creation. Despite the progress in large generative models, decomposing a single image into layers remains challenging due to limited methods and data. We observe a strong connection between layer decomposition and in/outpainting tasks, and propose adapting a diffusion-based inpainting model for layer decomposition using lightweight finetuning. To further preserve detail in the latent space, we introduce a novel multi-modal context fusion module with linear attention complexity. Our model is trained purely on a synthetic dataset constructed from open-source assets and achieves superior performance in object removal and occlusion recovery, unlocking new possibilities in downstream editing and creative applications.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。