用扩散 Transformer 模型分解半透明图像层,解决遮挡模糊与数据不足问题。
DiffDecompose: Layer-Wise Decomposition of Alpha-Composited Images via Diffusion Transformers
- 基于扩散 Transformer 的上下文分解机制,不依赖逐层标注
- 在 AlphaBlend 数据集上实现 6 种真实场景的层分解,准确率提升显著
- 适合图像修复、视觉编辑等需要精确透明层分离的应用
扩散模型在图像生成任务中表现卓越,但现有图像分解方法在处理半透明或透明层遮挡时受限于掩码先验依赖、静态物体假设及数据匮乏。本文提出全新任务:基于非线性遮挡的透明/半透明层图像分层分解。为应对层模糊、泛化难与数据少的问题,我们构建了首个大规模高质量数据集 AlphaBlend,支持六种真实子任务(如半透明光晕去除、半透明细胞分解、玻璃器皿分解)。在此基础上,提出 DiffDecompose 框架,通过条件扩散 Transformer 学习输入图像、语义提示和混合类型下的后验分布。不同于直接回归 alpha 蒙版,该方法采用上下文分解,无需每层监督即可预测一层或多层,并引入层位置编码克隆以保持跨层像素级对应。在 AlphaBlend 和公开 LOGO 数据集上的实验验证了其有效性。代码与数据将在论文接受后开源。
原文摘要 · Abstract (English)
Diffusion models have recently motivated great success in many generation tasks like object removal. Nevertheless, existing image decomposition methods struggle to disentangle semi-transparent or transparent layer occlusions due to mask prior dependencies, static object assumptions, and the lack of datasets. In this paper, we delve into a novel task: Layer-Wise Decomposition of Alpha-Composited Images, aiming to recover constituent layers from single overlapped images under the condition of semi-transparent/transparent alpha layer non-linear occlusion. To address challenges in layer ambiguity, generalization, and data scarcity, we first introduce AlphaBlend, the first large-scale and high-quality dataset for transparent and semi-transparent layer decomposition, supporting six real-world subtasks (e.g., translucent flare removal, semi-transparent cell decomposition, glassware decomposition). Building on this dataset, we present DiffDecompose, a diffusion Transformer-based framework that learns the posterior over possible layer decompositions conditioned on the input image, semantic prompts, and blending type. Rather than regressing alpha mattes directly, DiffDecompose performs In-Context Decomposition, enabling the model to predict one or multiple layers without per-layer supervision, and introduces Layer Position Encoding Cloning to maintain pixel-level correspondence across layers. Extensive experiments on the proposed AlphaBlend dataset and public LOGO dataset verify the effectiveness of DiffDecompose. The code and dataset will be available upon paper acceptance. Our code will be available at: https://github.com/Wangzt1121/DiffDecompose.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。