arXiv:2501.16177cs.CVcs.AI2025-01被引 3

让3D服装自动贴合人体,生成更真实可穿的虚拟服饰。

BAG: Body-Aligned 3D Wearable Asset Generation

  • 用人体姿态和形状控制生成过程,实现精准对齐
  • 在多视角图像上保持一致性,生成高质量3D资产
  • 适合虚拟试衣、游戏道具生成等需要贴合人体的应用

尽管近期3D形状生成模型取得显著进展,但如何自动生成可穿戴的3D资产仍属空白。本文提出BAG(Body-aligned Asset Generation),通过人体形状与姿态信息控制生成过程,实现3D可穿戴资产在目标人体上的自动适配。首先构建基于单图到一致多视角图像的扩散模型,并在大型Objaverse数据集上训练以保证多样性与泛化性;随后训练一个Controlnet,利用目标人体在标准空间中的多视角2D投影(像素值代表XYZ坐标)作为控制信号,引导多视角生成器产出体对齐的多视角图像。这些图像输入原生3D扩散模型生成资产3D形状。最后,通过多视角轮廓监督恢复相似变换,并借助物理模拟解决资产与身体穿透问题,实现精确贴合。实验表明,BAG在图像提示遵循能力、形状多样性和质量上均优于现有方法。

原文摘要 · Abstract (English)

While recent advancements have shown remarkable progress in general 3D shape generation models, the challenge of leveraging these approaches to automatically generate wearable 3D assets remains unexplored. To this end, we present BAG, a Body-aligned Asset Generation method to output 3D wearable asset that can be automatically dressed on given 3D human bodies. This is achived by controlling the 3D generation process using human body shape and pose information. Specifically, we first build a general single-image to consistent multiview image diffusion model, and train it on the large Objaverse dataset to achieve diversity and generalizability. Then we train a Controlnet to guide the multiview generator to produce body-aligned multiview images. The control signal utilizes the multiview 2D projections of the target human body, where pixel values represent the XYZ coordinates of the body surface in a canonical space. The body-conditioned multiview diffusion generates body-aligned multiview images, which are then fed into a native 3D diffusion model to produce the 3D shape of the asset. Finally, by recovering the similarity transformation using multiview silhouette supervision and addressing asset-body penetration with physics simulators, the 3D asset can be accurately fitted onto the target human body. Experimental results demonstrate significant advantages over existing methods in terms of image prompt-following capability, shape diversity, and shape quality. Our project page is available at https://bag-3d.github.io/.

3D生成虚拟试衣扩散模型人体对齐

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。