用2D时尚数据提升单图3D人体纹理还原精度
FAMOUS: High-Fidelity Monocular 3D Human Digitization Using View Synthesis
- 利用2D时尚数据学习被遮挡背部的纹理
- 通过域对齐策略融合2D先验与输入图像
- 在标准基准上实现更真实的人体纹理与几何
深度隐式建模与姿态模型的进步显著提升了仅从单张图像生成3D人体的能力。尽管当前顶尖方法在几何精度上已大幅改善，但在推断纹理方面仍面临挑战，尤其是在正视图中被遮挡的背部区域。这一局限主要源于大规模多样化3D数据集稀缺，而2D数据集则丰富且易获取。为此，本文提出利用广泛的2D时尚数据集来增强3D人体数字化中的纹理与形状预测。通过引入2D时尚数据的先验信息，并结合所提出的域对齐策略，学习被遮挡的后视纹理，再与输入图像融合，生成完整纹理的3D网格。在多个标准3D人体基准上的大量实验表明，该方法在纹理和几何表现上均优于现有方法。代码与数据集见：https://github.com/humansensinglab/FAMOUS。
原文摘要 · Abstract (English)
The advancement in deep implicit modeling and articulated models has significantly enhanced the process of digitizing human figures in 3D from just a single image. While state-of-the-art methods have greatly improved geometric precision, the challenge of accurately inferring texture remains, particularly in obscured areas such as the back of a person in frontal-view images. This limitation in texture prediction largely stems from the scarcity of large-scale and diverse 3D datasets, whereas their 2D counterparts are abundant and easily accessible. To address this issue, our paper proposes leveraging extensive 2D fashion datasets to enhance both texture and shape prediction in 3D human digitization. We incorporate 2D priors from the fashion dataset to learn the occluded back view, refined with our proposed domain alignment strategy. We then fuse this information with the input image to obtain a fully textured mesh of the given person. Through extensive experimentation on standard 3D human benchmarks, we demonstrate the superior performance of our approach in terms of both texture and geometry. Code and dataset is available at https://github.com/humansensinglab/FAMOUS.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。