arXiv:2509.26621cs.CV2025-09被引 2

用少量图像重建穿衣服的人体,还能生成逼真新视角画面。

HART: Human Aligned Reconstruction Transformer

  • 通过自监督预测3D点、法向和人体对应关系,实现稀疏视角重建。
  • 在2.3K合成数据上训练,多项指标领先现有方法15%-27%。
  • 适合做真实场景下人体建模与渲染,尤其关注衣物和遮挡问题。

我们提出HART,一种统一的稀疏视角人体重建框架。输入为少量未标定的彩色图像,输出包括完整闭合的穿衣网格、对齐的SMPL-X人体模型以及用于逼真新视角渲染的高斯溅射表示。以往方法要么优化参数化模板(忽略松散衣物和人-物交互),要么在简化相机假设下训练隐式函数,难以应用于真实场景。HART预测每像素的3D点图、法线和人体对应关系,采用遮挡感知的泊松重建恢复完整几何,包括自遮挡区域。这些预测与参数化SMPL-X模型对齐,确保结构合理性同时捕捉松散衣物和交互。重建网格用于初始化高斯溅射,进一步支持稀疏视角渲染。尽管仅在2.3K合成扫描上训练,HART在多个数据集上达到领先性能:穿衣网格的Chamfer Distance提升18-23%,SMPL-X估计的PA-V2V下降6-27%,新视角合成的LPIPS降低15-27%。结果表明,前馈变换器可作为真实世界中鲁棒人体重建的可扩展模型。代码与模型将公开。

原文摘要 · Abstract (English)

We introduce HART, a unified framework for sparse-view human reconstruction. Given a small set of uncalibrated RGB images of a person as input, it outputs a watertight clothed mesh, the aligned SMPL-X body mesh, and a Gaussian-splat representation for photorealistic novel-view rendering. Prior methods for clothed human reconstruction either optimize parametric templates, which overlook loose garments and human-object interactions, or train implicit functions under simplified camera assumptions, limiting applicability in real scenes. In contrast, HART predicts per-pixel 3D point maps, normals, and body correspondences, and employs an occlusion-aware Poisson reconstruction to recover complete geometry, even in self-occluded regions. These predictions also align with a parametric SMPL-X body model, ensuring that reconstructed geometry remains consistent with human structure while capturing loose clothing and interactions. These human-aligned meshes initialize Gaussian splats to further enable sparse-view rendering. While trained on only 2.3K synthetic scans, HART achieves state-of-the-art results: Chamfer Distance improves by 18-23 percent for clothed-mesh reconstruction, PA-V2V drops by 6-27 percent for SMPL-X estimation, LPIPS decreases by 15-27 percent for novel-view synthesis on a wide range of datasets. These results suggest that feed-forward transformers can serve as a scalable model for robust human reconstruction in real-world settings. Code and models will be released.

人体重建图像生成3D建模视觉渲染

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。