实现任意视角的高保真虚拟试衣，让服装穿搭效果更真实。
VTON 360: High-Fidelity Virtual Try-On from Any Viewing Direction
- 将2D虚拟试衣扩展为多视角输入，保证不同角度看都一致。
- 引入法向图姿态表示和多视角注意力机制，提升3D一致性。
- 适合电商、时尚设计领域，需3D人体建模基础的开发者参考。
虚拟试衣（VTON）是电商与时尚设计中的变革性技术，可实现服装在人体上的逼真数字展示。本文提出 VTON 360，一种新型3D虚拟试衣方法，解决任意视角渲染下的高保真难题。我们利用3D模型与其多视角2D图像间的等价关系，将3D VTON重构为2D VTON的扩展，确保多视角结果的一致性。具体地，将2D VTON模型拓展为接收多视角服装与无服装人体图像作为输入，并提出三项新策略：i) 使用来自SMPL-X 3D人体模型的法向图构建伪3D姿态表示；ii) 设计多视角空间注意力机制，建模不同视角特征间的关联；iii) 引入多视角CLIP嵌入，结合相机信息增强服装的CLIP特征表达。在大规模真实数据集及电商平台服装图像上进行的大量实验验证了该方法的有效性。项目主页：https://scnuhealthy.github.io/VTON360。
原文摘要 · Abstract (English)
Virtual Try-On (VTON) is a transformative technology in e-commerce and fashion design, enabling realistic digital visualization of clothing on individuals. In this work, we propose VTON 360, a novel 3D VTON method that addresses the open challenge of achieving high-fidelity VTON that supports any-view rendering. Specifically, we leverage the equivalence between a 3D model and its rendered multi-view 2D images, and reformulate 3D VTON as an extension of 2D VTON that ensures 3D consistent results across multiple views. To achieve this, we extend 2D VTON models to include multi-view garments and clothing-agnostic human body images as input, and propose several novel techniques to enhance them, including: i) a pseudo-3D pose representation using normal maps derived from the SMPL-X 3D human model, ii) a multi-view spatial attention mechanism that models the correlations between features from different viewing angles, and iii) a multi-view CLIP embedding that enhances the garment CLIP features used in 2D VTON with camera information. Extensive experiments on large-scale real datasets and clothing images from e-commerce platforms demonstrate the effectiveness of our approach. Project page: https://scnuhealthy.github.io/VTON360.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。