arXiv:2411.03047cs.CVcs.GR2024-11被引 20

用高质量细节分层数据集,让单图生成高保真服装更鲁棒。

GarVerseLOD: High-Fidelity 3D Garment Reconstruction from a Single In-the-Wild Image using a Dataset with Levels of Details

  • 构建分层级细节数据集,拆解复杂重建任务
  • 6000个专业制作的服装模型,支持像素级细节还原
  • 基于扩散模型生成真实感配对图像,提升泛化能力

神经隐式函数在从多视角甚至单视角图像重建衣着人体方面取得显著进展。然而,现有方法在处理复杂衣物形变和身体姿态的未见图像时仍存在泛化困难。本文提出GarVerseLOD,一个新数据集与框架,旨在实现单张自然场景图像下前所未有的高保真3D服装重建鲁棒性。受大型生成模型成功启发,我们认为解决泛化挑战的关键在于3D服装数据的数量与质量。为此,GarVerseLOD收集了6000个由专业艺术家手工创建的高质量服装模型,包含精细几何细节。此外,我们发现将几何粒度解耦为不同层次细节(LOD)有助于提升模型泛化能力和推理精度。因此,该数据集从无细节风格化形状到与姿态融合且像素对齐的细节服装,形成层级结构,使高度欠约束问题可解。通过将推断分解为多个更易处理的任务,缩小搜索空间。为确保对真实场景图像的良好泛化,我们提出一种基于条件扩散模型的新标注范式,为每个服装模型生成大量高保真配对图像。我们在海量真实图像上评估该方法,实验表明,GarVerseLOD生成的独立服装部件质量显著优于先前方法。

原文摘要 · Abstract (English)

Neural implicit functions have brought impressive advances to the state-of-the-art of clothed human digitization from multiple or even single images. However, despite the progress, current arts still have difficulty generalizing to unseen images with complex cloth deformation and body poses. In this work, we present GarVerseLOD, a new dataset and framework that paves the way to achieving unprecedented robustness in high-fidelity 3D garment reconstruction from a single unconstrained image. Inspired by the recent success of large generative models, we believe that one key to addressing the generalization challenge lies in the quantity and quality of 3D garment data. Towards this end, GarVerseLOD collects 6,000 high-quality cloth models with fine-grained geometry details manually created by professional artists. In addition to the scale of training data, we observe that having disentangled granularities of geometry can play an important role in boosting the generalization capability and inference accuracy of the learned model. We hence craft GarVerseLOD as a hierarchical dataset with levels of details (LOD), spanning from detail-free stylized shape to pose-blended garment with pixel-aligned details. This allows us to make this highly under-constrained problem tractable by factorizing the inference into easier tasks, each narrowed down with smaller searching space. To ensure GarVerseLOD can generalize well to in-the-wild images, we propose a novel labeling paradigm based on conditional diffusion models to generate extensive paired images for each garment model with high photorealism. We evaluate our method on a massive amount of in-the-wild images. Experimental results demonstrate that GarVerseLOD can generate standalone garment pieces with significantly better quality than prior approaches. Project page: https://garverselod.github.io/

3D重建服装生成数据集扩散模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。