arXiv:2507.09308cs.CVcs.AI2025-07被引 7

首个统一生成透明图像的端到端模型,仅用8千张图就超越此前方法。

AlphaVAE: Unified End-to-End RGBA Image Reconstruction and Generation with Alpha-Aware Representation Learning

  • 在预训练RGB VAE基础上新增独立透明通道,实现RGBA联合建模。
  • 仅用8000张图像训练,重建性能比现有方法提升4.9 dB PSNR、3.2% SSIM。
  • 开源数据集与代码,适合需要透明图像生成的研究者使用。

最近的潜在扩散模型通过利用预训练VAE,在低计算成本下实现了高质量的RGB图像合成。然而,由于缺乏大规模基准,透明或分层内容(RGBA图像)的生成仍鲜有研究。本文提出ALPHA,首个全面的RGBA基准,通过在标准背景上进行透明度混合,将常规RGB评估指标拓展至四通道图像。我们进一步提出ALPHAVAE,一种统一的端到端RGBA VAE,通过在预训练RGB VAE基础上增加专用透明通道实现扩展。该模型采用复合目标函数,包含透明度混合像素重建、块级保真度、感知一致性及双重KL散度约束,以确保RGB与透明度表示在潜在空间中的保真度。我们的模型仅用8000张图像训练,相比先前方法在重建上提升4.9 dB PSNR和3.2% SSIM,同时在微调至潜在扩散框架后展现出更优的透明图像生成能力。代码、数据与模型已公开于https://github.com/o0o0o00o0/AlphaVAE。

原文摘要 · Abstract (English)

Recent advances in latent diffusion models have achieved remarkable results in high-fidelity RGB image synthesis by leveraging pretrained VAEs to compress and reconstruct pixel data at low computational cost. However, the generation of transparent or layered content (RGBA image) remains largely unexplored, due to the lack of large-scale benchmarks. In this work, we propose ALPHA, the first comprehensive RGBA benchmark that adapts standard RGB metrics to four-channel images via alpha blending over canonical backgrounds. We further introduce ALPHAVAE, a unified end-to-end RGBA VAE that extends a pretrained RGB VAE by incorporating a dedicated alpha channel. The model is trained with a composite objective that combines alpha-blended pixel reconstruction, patch-level fidelity, perceptual consistency, and dual KL divergence constraints to ensure latent fidelity across both RGB and alpha representations. Our RGBA VAE, trained on only 8K images in contrast to 1M used by prior methods, achieves a +4.9 dB improvement in PSNR and a +3.2% increase in SSIM over LayerDiffuse in reconstruction. It also enables superior transparent image generation when fine-tuned within a latent diffusion framework. Our code, data, and models are released on https://github.com/o0o0o00o0/AlphaVAE for reproducibility.

图像生成透明图像VAE扩散模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。