arXiv:2411.17864cs.CV2024-11CVPR被引 36

让图像分层编辑更精准,自动分离背景与带透明效果的前景。

Generative Image Layer Decomposition with Visual Effects

  • 基于扩散模型构建分层分解框架,支持独立编辑各图层。
  • 在多个基准上优于现有方法,用户测试中显著提升编辑质量。
  • 适合需要精细图像编辑的设计师、内容创作者使用。

近年来,基于扩散的大规模生成模型显著提升了图像编辑能力,但精确控制图像构成仍具挑战。分层表示可实现对图像组件的独立编辑,是用户驱动内容创作的关键,然而现有方法难以将图像分解为合理图层并准确保留阴影、反射等透明视觉效果。本文提出《LayerDecomp》——一种生成式图像分层分解框架,可输出逼真的干净背景与高质量透明前景,并忠实还原视觉效果。为支持有效训练,我们设计了自动扩增模拟多层数据的管道,合成各类视觉效果;同时补充真实拍摄含自然视觉效果的图像以增强泛化性。此外,提出一致性损失,在无真实标注时仍能学习透明前景的准确表征。实验表明,该方法在物体移除与空间编辑任务中全面超越现有方法,多个基准测试及用户研究验证其优越性,为分层图像编辑开辟了全新创作可能。

原文摘要 · Abstract (English)

Recent advancements in large generative models, particularly diffusion-based methods, have significantly enhanced the capabilities of image editing. However, achieving precise control over image composition tasks remains a challenge. Layered representations, which allow for independent editing of image components, are essential for user-driven content creation, yet existing approaches often struggle to decompose image into plausible layers with accurately retained transparent visual effects such as shadows and reflections. We propose $\textbf{LayerDecomp}$, a generative framework for image layer decomposition which outputs photorealistic clean backgrounds and high-quality transparent foregrounds with faithfully preserved visual effects. To enable effective training, we first introduce a dataset preparation pipeline that automatically scales up simulated multi-layer data with synthesized visual effects. To further enhance real-world applicability, we supplement this simulated dataset with camera-captured images containing natural visual effects. Additionally, we propose a consistency loss which enforces the model to learn accurate representations for the transparent foreground layer when ground-truth annotations are not available. Our method achieves superior quality in layer decomposition, outperforming existing approaches in object removal and spatial editing tasks across several benchmarks and multiple user studies, unlocking various creative possibilities for layer-wise image editing. The project page is https://rayjryang.github.io/LayerDecomp.

图像编辑分层分解扩散模型透明效果

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。