用知识迁移实现前景与背景无缝融合的图像分层生成
BFS: Back-to-Front Layered Image Synthesis via Knowledge Transfer

- 双分支扩散框架实现前后景双向知识传递
- 两阶段训练提升前景质量,克服数据稀缺问题
- 支持阴影反射等视觉效果合成,适合创意编辑场景
随着生成模型拓展视觉内容创作边界,分层图像合成成为可控且具创造力的编辑方向。然而现有方法存在局限:基于分解的方法难以实现清晰分离,基于生成的方法则面临训练数据获取难、质量与场景多样性低的问题。本文提出BFS,一种新型生成式分层图像合成框架。给定背景图和用户引导,BFS不仅合成前景物体,还包含其关联的阴影、反光等视觉效果,并与背景自然融合生成一致的合成图像。为克服数据稀缺并提升前景质量,我们利用相对易学的非分层图像合成知识进行前景生成。采用双分支扩散框架,两个互连分支分别生成复合图和前景层,实现双向知识迁移。基于此,设计两阶段训练方案,借助高质量非分层复合图像数据集有效提升前景质量。大量实验(含用户研究)表明,BFS生成的分层图像质量显著优于现有方法。
原文摘要 · Abstract (English)
As generative models expand the possibilities of visual content creation, layered image synthesis has emerged as a promising direction for controllable and creative editing. However, existing methods struggle to fully realize this potential. Decomposition-based methods often struggle with clean separation, while generation-based methods suffer from difficulty in training data acquisition, reducing quality and scene diversity. In this paper, we propose BFS, a novel generation-based framework for layered image synthesis. Specifically, given a background image and user guidance, BFS synthesizes a foreground layer that incorporates not only a foreground object but also its associated visual effects, such as shadows and reflections, while seamlessly harmonizing with the background to produce a coherent composite. To enable diverse and high-quality foreground layer synthesis while overcoming data scarcity, we leverage the comparatively easy-to-learn knowledge of unlayered image synthesis for the foreground synthesis. To this end, we adopt a dual-branch diffusion framework in which two interconnected branches generate a composite image and a foreground layer, respectively, enabling bidirectional knowledge transfer. Based on this framework, we propose a two-stage training scheme that utilizes a high-quality unlayered composite image dataset to effectively enhance foreground quality. Extensive experiments, including a user study, show that BFS produces high-quality layered images, consistently outperforming prior methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。