基于先验引导学习，实现单目头戴摄像头下全身网格高精度重建。
Egocentric Whole-Body Human Mesh Recovery with Prior-Guided Learning
- 利用优化生成的伪真值与多先验融合，提升人体网格重建质量。
- 在多个基准上优于现有方法，尤其在手部和面部细节恢复上表现显著。
- 适用于增强现实/虚拟现实中的真实场景人体建模，适合研究者复现。
单目头戴摄像头下的自指人体网格重建（HMR）在增强现实/虚拟现实应用中日益重要，但因缺乏基于参数化人体模型（如SMPL、SMPL-X）的真实标注数据而面临挑战。现有方法通常依赖回归生成的伪真值，且侧重姿态估计，难以恢复手部、面部等精细结构。本文提出一种先验引导的学习框架，从单张自指图像中重建完整人体网格。构建了与3D关节监督对齐的更精确优化型伪真值，并通过适配外视角HMR基础模型及扩散模型姿态先验引入多重先验信息。进一步采用确定性去畸变模块处理自指图像中的鱼眼畸变。在多个自指基准上的实验表明，本方法在全身重建性能上超越当前最优方法，且优化型伪真值比传统回归型伪真值显著更准确。为促进可复现性，代码与数据标注已公开于https://github.com/naso06/EgoSMPLX。
原文摘要 · Abstract (English)
Egocentric human mesh recovery (HMR) from monocular head-mounted cameras is increasingly important for AR/VR applications, but remains challenging due to the lack of reliable ground-truth (GT) annotations based on parametric human body models such as SMPL and SMPL-X for real egocentric images. Existing egocentric HMR methods typically rely on pseudo-GT and focus on body pose estimation, which limits their ability to recover fine-grained whole-body details such as hands and face. We study egocentric whole-body human mesh recovery and propose a prior-guided learning framework that reconstructs whole-body meshes from a single egocentric image. We construct more accurate optimization-based pseudo-GT aligned with 3D joint supervision, and leverage multiple priors by adapting an exocentric HMR foundation model together with a diffusion-based pose prior. A deterministic undistortion module is further adopted to handle fisheye distortions in egocentric images. Experiments across multiple egocentric benchmarks demonstrate improved whole-body reconstruction compared to state-of-the-art methods, and show that our optimization-based pseudo-GT is substantially more accurate than existing regression-based pseudo-GT. To facilitate reproducibility, the code and dataset annotations are publicly available at https://github.com/naso06/EgoSMPLX.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。