用轻量级扩散变压器实现高效零样本物体定制，速度更快内存更省。
E-MD3C: Taming Masked Diffusion Transformers for Efficient Zero-Shot Object Customization
- 用掩码扩散Transformer处理图像潜在特征，替代耗资源的Unet。
- 参数仅1/4却提速2.5倍，显存降低2/3，性能优于现有方法。
- 适合追求快速部署和低资源推理的生成模型应用者。
我们提出E-MD3C（高效掩码扩散Transformer与解耦条件及紧凑收集器），一种高效的零样本物体图像定制框架。不同于依赖高资源Unet架构的先前工作，本方法采用轻量级掩码扩散Transformer在潜在块上操作，显著提升计算效率。框架包含三个核心组件：(1) 用于处理自编码器潜在表示的高效掩码扩散Transformer；(2) 解耦条件设计，兼顾紧凑性与背景对齐、细节保留；(3) 可学习的条件收集器，将多输入合并为紧凑表示以实现高效去噪与学习。在VITON-HD数据集上，E-MD3C在PSNR、FID、SSIM和LPIPS等指标上超越现有方法。仅需1720M Unet模型1/4的参数（468M），推理速度提升2.5倍，显存使用减少2/3。
原文摘要 · Abstract (English)
We propose E-MD3C ($\underline{E}$fficient $\underline{M}$asked $\underline{D}$iffusion Transformer with Disentangled $\underline{C}$onditions and $\underline{C}$ompact $\underline{C}$ollector), a highly efficient framework for zero-shot object image customization. Unlike prior works reliant on resource-intensive Unet architectures, our approach employs lightweight masked diffusion transformers operating on latent patches, offering significantly improved computational efficiency. The framework integrates three core components: (1) an efficient masked diffusion transformer for processing autoencoder latents, (2) a disentangled condition design that ensures compactness while preserving background alignment and fine details, and (3) a learnable Conditions Collector that consolidates multiple inputs into a compact representation for efficient denoising and learning. E-MD3C outperforms the existing approach on the VITON-HD dataset across metrics such as PSNR, FID, SSIM, and LPIPS, demonstrating clear advantages in parameters, memory efficiency, and inference speed. With only $\frac{1}{4}$ of the parameters, our Transformer-based 468M model delivers $2.5\times$ faster inference and uses $\frac{2}{3}$ of the GPU memory compared to an 1720M Unet-based latent diffusion model.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。