高效虚拟试穿模型,生成细节清晰、速度快的试穿图像。
EfficientVITON: An Efficient Virtual Try-On Model using Optimized Diffusion Process
- 用优化扩散过程+空间编码器,保留衣物细节并贴合人体形态。
- 在VITON-HD数据集上达到当前最优效果,生成时间大幅缩短。
- 适合电商、时尚应用,兼顾质量与部署效率,无需配对数据。
虚拟试穿能让用户仅通过镜像即可数字试穿服装,核心挑战在于实现逼真的图像到图像转换,使衣物能适配多样人体形态、姿势和体型。早期基于2D变换的方法虽快但画质不佳;基于GAN的方法虽提升真实感,却依赖成对数据;更灵活的方法虽视觉效果好,但计算开销大、耗时长。扩散模型虽有高保真潜力,现有虚拟试穿工具仍存在细节丢失与形变问题。为此,本文提出EfficientVITON,利用预训练的Stable Diffusion模型提升图像质量与部署可行性。系统引入空间编码器以保持衣物细节,并设计零交叉注意力块捕捉衣物贴合人体的细微特征。输入图像经精心预处理,扩散过程被优化以显著减少生成时间,且不损失图像质量。训练分两阶段进行,通过平衡多种损失函数确保试穿准确性和图像高质量。在VITON-HD数据集上的严格测试及真实场景验证表明,EfficientVITON达到当前最佳性能。
原文摘要 · Abstract (English)
Would not it be much more convenient for everybody to try on clothes by only looking into a mirror ? The answer to that problem is virtual try-on, enabling users to digitally experiment with outfits. The core challenge lies in realistic image-to-image translation, where clothing must fit diverse human forms, poses, and figures. Early methods, which used 2D transformations, offered speed, but image quality was often disappointing and lacked the nuance of deep learning. Though GAN-based techniques enhanced realism, their dependence on paired data proved limiting. More adaptable methods offered great visuals but demanded significant computing power and time. Recent advances in diffusion models have shown promise for high-fidelity translation, yet the current crop of virtual try-on tools still struggle with detail loss and warping issues. To tackle these challenges, this paper proposes EfficientVITON, a new virtual try-on system leveraging the impressive pre-trained Stable Diffusion model for better images and deployment feasibility. The system includes a spatial encoder to maintain clothings finer details and zero cross-attention blocks to capture the subtleties of how clothes fit a human body. Input images are carefully prepared, and the diffusion process has been tweaked to significantly cut generation time without image quality loss. The training process involves two distinct stages of fine-tuning, carefully incorporating a balance of loss functions to ensure both accurate try-on results and high-quality visuals. Rigorous testing on the VITON-HD dataset, supplemented with real-world examples, has demonstrated that EfficientVITON achieves state-of-the-art results.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。