解决多人姿态恢复中的深度冲突问题,实现场景一致的精确建模。
Towards Metric-Aware Multi-Person Mesh Recovery by Jointly Optimizing Human Crowd in Camera Space
- 通过联合优化人群相机空间位置,利用身高与单目深度先验
- 构建包含56万张图像的高质量多人群数据集,平均每图4.8人
- 端到端网络直接输出度量尺度下的三维人体网格,适合真实场景应用
从单张图像进行多人人体网格重建是一项挑战性任务,受限于真实场景训练数据的匮乏。现有伪真值生成方法多为单人中心,未对人群整体进行联合优化,导致同一图像中个体间存在深度和尺度矛盾。为此,我们提出深度条件平移优化(DTO),一种基于优化的方法,联合修正人群中所有个体在相机空间的平移。该方法结合人体身高先验和单目深度估计器提供的深度线索,在最大后验框架下求解场景一致的人体布局。将DTO应用于4D-Humans数据集,构建了新数据集DTO-Humans,包含0.56M张高质量、场景一致的多人图像,平均每张图像含4.8人。此外,提出度量感知人体重建网络(Metric-Aware HMR),通过相机分支与相对度量损失,直接输出度量尺度下的人体网格。大量实验表明,该方法在相对深度推理和人体网格重建上达到领先性能。代码已开源。
原文摘要 · Abstract (English)
Multi-person human mesh recovery from a single image is a challenging task, hindered by the scarcity of in-the-wild training data. Prevailing in-the-wild human mesh pseudo-ground-truth (pGT) generation pipelines are single-person-centric, where each human is processed individually without joint optimization. This oversight leads to a lack of scene-level consistency, producing individuals with conflicting depths and scales within the same image. To address this, we introduce Depth-conditioned Translation Optimization (DTO), a novel optimization-based method that jointly refines the camera-space translations of all individuals in a crowd. By leveraging anthropometric priors on human height and depth cues from a monocular depth estimator, DTO solves for a scene-consistent placement of all subjects within a principled Maximum a posteriori (MAP) framework. Applying DTO to the 4D-Humans dataset, we construct DTO-Humans, a new large-scale pGT dataset of 0.56M high-quality, scene-consistent multi-person images, featuring dense crowds with an average of 4.8 persons per image. Furthermore, we propose Metric-Aware HMR, an end-to-end network that directly estimates human mesh and camera parameters in metric scale. This is enabled by a camera branch and a relative metric loss that enforces plausible relative scales. Extensive experiments demonstrate that our method achieves state-of-the-art performance on relative depth reasoning and human mesh recovery. Code is available at: https://github.com/gouba2333/MA-HMR.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。