统一生成透明度信息,让图像合成更自然真实。
OmniAlpha: Aligning Transparency-Aware Generation via Multi-Task Unified Reinforcement Learning
- 用强化学习联合优化多任务透明度生成
- 在层分解任务上比基线低9.07%的RGB误差
- 适合需要精细透明效果的图像编辑场景
透明度感知生成需同时建模RGB外观、基于alpha的不透明度及跨层合成,这对图像抠图、物体移除、图层分解和多层内容创作至关重要。现有方法多为孤立任务设计,缺乏统一框架。尽管统一模型理想,但仅靠监督微调不足以优化组合保真度、α边界精度和结构一致性。为此,我们提出OmniAlpha,一种基于多任务统一强化学习的透明度感知生成框架。它结合端到端的alpha感知变分自编码器与序列到序列扩散Transformer,采用双向层轴位置编码,在一次前向传播中联合建模多个RGBA输入输出。基于多任务SFT冷启动,进一步通过类似GRPO的后训练,使用解码后RGBA输出定义的层感知奖励,直接优化跨层一致性与精细透明细节。在五类透明度相关任务上的实验表明,OmniAlpha持续优于统一SFT基线,并达到甚至超过专用专家模型性能,如在层分解任务上实现9.07%的相对RGB L1降低,自动抠图任务中在SAD/Grad指标上分别较传统工具提升74%/68%。
原文摘要 · Abstract (English)
Transparency-aware generation requires modeling not only RGB appearance but also alpha-based opacity and cross-layer composition, which are essential for tasks such as image matting, object removal, layer decomposition, and multi-layer content creation. However, existing RGBA-related methods remain largely fragmented, with separate pipelines designed for individual tasks. While a unified model is desirable, supervised fine-tuning alone is insufficient, as localized regression objectives cannot directly optimize the compositional fidelity, alpha-boundary precision, and structural consistency required for high-quality RGBA generation. To address this, we propose OmniAlpha, a unified multi-task reinforcement learning framework for transparency-aware generation and manipulation. OmniAlpha combines an end-to-end alpha-aware VAE and a sequence-to-sequence Diffusion Transformer, with a bi-directional layer axis in positional encoding to jointly model multiple RGBA inputs and outputs within a single forward pass. Built on a multi-task SFT cold start, it further performs GRPO-style post-training with layer-aware rewards defined on decoded RGBA outputs, enabling direct optimization of cross-layer coherence and fine transparency details. Experiments across five categories of transparency-aware tasks show that OmniAlpha consistently outperforms its unified SFT baseline and achieves strong performance against specialized expert models, including a 9.07% relative reduction in RGB L1 on layer decomposition and 74%/68% improvements over conventional matting tools on SAD/Grad for automatic matting.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。