从一张照片生成可模拟的分离3D服装，支持虚拟试穿动画
Dress-1-to-3: Single Image to Simulation-Ready 3D Outfit with Diffusion Prior and Differentiable Physics
- 结合扩散模型与可微物理仿真，重建带缝制线的分离3D服装
- 优化后服装与输入图像的几何对齐度显著提升，满足动态模拟需求
- 适合需要真实动态服装动画的虚拟试穿、数字人应用
近期大型模型的发展显著推动了图像到3D的重建技术。然而，生成的模型通常融合为整体，限制了在下游任务中的应用。本文聚焦于3D服装生成，这是虚拟试穿和动态服装动画等应用的关键。我们提出Dress-1-to-3，一个新颖的端到端流程，能从一张自然场景图像中重建出具有真实缝制线、可模拟的分离3D服装及人体。方法首先利用预训练的图像到缝制线生成模型获得粗略缝制线，再通过预训练的多视角扩散模型生成多视角图像。随后，基于生成的多视角图像，使用基于可微物理的服装仿真器对缝制线进行精细化调整。大量实验表明，该优化方法显著提升了重建3D服装与人体在几何上与输入图像的一致性。进一步地，结合纹理生成模块与人体运动生成模块，实现了定制化、物理合理的动态服装演示。项目页面：https://dress-1-to-3.github.io/
原文摘要 · Abstract (English)
Recent advances in large models have significantly advanced image-to-3D reconstruction. However, the generated models are often fused into a single piece, limiting their applicability in downstream tasks. This paper focuses on 3D garment generation, a key area for applications like virtual try-on with dynamic garment animations, which require garments to be separable and simulation-ready. We introduce Dress-1-to-3, a novel pipeline that reconstructs physics-plausible, simulation-ready separated garments with sewing patterns and humans from an in-the-wild image. Starting with the image, our approach combines a pre-trained image-to-sewing pattern generation model for creating coarse sewing patterns with a pre-trained multi-view diffusion model to produce multi-view images. The sewing pattern is further refined using a differentiable garment simulator based on the generated multi-view images. Versatile experiments demonstrate that our optimization approach substantially enhances the geometric alignment of the reconstructed 3D garments and humans with the input image. Furthermore, by integrating a texture generation module and a human motion generation module, we produce customized physics-plausible and realistic dynamic garment demonstrations. Project page: https://dress-1-to-3.github.io/
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。