arXiv:2601.00328cs.CV2026-01

用统一潜空间的桥式扩散模型,从单张图重建高保真人体几何与外观。

Joint Geometry-Appearance Human Reconstruction in a Unified Latent Space via Bridge Diffusion

  • 将几何与外观统一为3D高斯表示,通过共享稀疏VAE压缩至统一潜空间。
  • 采用桥式扩散,从部分观测中逐步推断缺失的潜变量,提升重建一致性。
  • 适用于真实场景,优于当前最先进方法,在野外条件下表现更优。

从单张RGB图像实现高保真、一致的人体三维数字人几何与外观重建是一项固有挑战。现有方法多采用解耦流程分别处理几何估计与外观合成,常导致统一重建困难和不一致问题。本文提出JGA-LBD框架,将几何与外观建模统一到联合潜空间,并将生成过程形式化为桥式扩散。鉴于直接融合异构输入条件(如深度图、SMPL模型)带来巨大训练难度,我们将其统一为3D高斯表示,再通过共享稀疏变分自编码器(VAE)压缩至统一潜空间。随后,专用桥式扩散从目标潜码的部分观测出发,仅聚焦于推断缺失成分。最后,专用解码模块从推断出的潜表示中提取完整人体几何结构并渲染新视角。实验表明,JGA-LBD在几何保真度与外观质量上均优于当前最先进方法,包括具有挑战性的野外场景。代码将公开于https://github.com/haiantyz/JGA-LBD。

原文摘要 · Abstract (English)

Achieving consistent and high-fidelity geometry and appearance reconstruction of 3D digital humans from a single RGB image is inherently a challenging task. Existing studies typically resort to decoupled pipelines for geometry estimation and appearance synthesis, often hindering unified reconstruction and causing inconsistencies. This paper introduces \textbf{JGA-LBD}, a novel framework that unifies the modeling of geometry and appearance into a joint latent representation and formulates the generation process as bridge diffusion. Observing that directly integrating heterogeneous input conditions (e.g., depth maps, SMPL models) leads to substantial training difficulties, we unify all conditions into the 3D Gaussian representations, which can be further compressed into a unified latent space through a shared sparse variational autoencoder (VAE). Subsequently, the specialized form of bridge diffusion enables to start with a partial observation of the target latent code and solely focuses on inferring the missing components. Finally, a dedicated decoding module extracts the complete 3D human geometric structure and renders novel views from the inferred latent representation. Experiments demonstrate that JGA-LBD outperforms current state-of-the-art approaches in terms of both geometry fidelity and appearance quality, including challenging in-the-wild scenarios. Our code will be made publicly available at https://github.com/haiantyz/JGA-LBD.

3D重建扩散模型人体建模

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。