一次性恢复全年龄段多人3D人体网格,解决深度模糊问题。
Anny-Fit: All-Age Human Mesh Recovery

- 直接在相机坐标系中联合优化多人,保持空间一致性。
- 2D重投影误差提升13-16,3D估计误差降低9-29。
- 利用视觉语言模型生成年龄性别等语义信息,适合真实场景应用。
从单张图像恢复3D人体姿态与形状是人体感知的核心任务,但现有方法多假设对象为成人且独立优化每人。这在真实世界全年龄段场景中失效,因身体比例与深度需协同求解。本文提出Anny-Fit,一种多人群体、相机空间优化框架,用于全年龄段3D人体网格重建(HMR)。不同于传统逐人拟合方法,Anny-Fit直接在相机坐标系中联合优化所有人,强制全局空间一致性。核心在于融合多种专家知识:度量深度图、实例分割、2D关键点,以及由视觉语言模型(VLM)推导的语义属性(如年龄、性别),均由独立预训练网络获取。这些互补信号共同引导优化,缓解全年龄段场景中的深度尺度模糊问题。在多个数据集上,Anny-Fit显著提升2D重投影精度(+13至+16)、相对深度排序准确率(+6至+7)、3D估计误差(-9至-29)及形状估计性能(+25至+82),生成更一致的场景。最后,我们证明可通过Anny-Fit在训练数据上生成伪真值标注,将VLM语义知识蒸馏进HMR模型,使其学习语义有意义的形状参数并提升性能。本方法通过零样本迁移,使成人生长的HMR流水线无需重训即可适配全年龄谱。代码已开源于https://github.com/naver/anny-fit。
原文摘要 · Abstract (English)
Recovering 3D human pose and shape from a single image remains a cornerstone of human-centric vision, yet most methods assume adult subjects and optimize each person independently. These assumptions fail in real-world, all-age scenes, where body proportions and depth must be resolved jointly. We introduce Anny-Fit, a multi-person, camera-space optimization framework for all-age 3D human mesh recovery (HMR). Unlike existing per-person fitting methods, Anny-Fit jointly optimizes all individuals directly in the camera coordinate system, enforcing global spatial consistency. At the core of our approach is the use of multiple forms of expert knowledge -- including metric depth maps, instance segmentation, 2D keypoints, and, VLM-derived semantic attributes such as age and gender -- each obtained from dedicated off-the-shelf networks. These complementary signals jointly guide the optimization, constraining the depth-scale ambiguity characteristic of all-age scenes. Across diverse datasets, Anny-Fit consistently improves 2D reprojection accuracy (+13 to 16), relative depth ordering (+6 to 7), 3D estimation error (-9 to -29) and shape estimation (+25 to +82), producing more coherent scenes. Finally, we show that VLM-based semantic knowledge can be distilled into an HMR model via the pseudo-ground-truth annotations produced by Anny-Fit on training data, enabling it to learn semantically meaningful shape parameters while improving HMR performance. Our approach bridges adult-only and all-age modeling by enabling zero-shot adaptation of adult-trained HMR pipelines to the full age spectrum without retraining. Code is publicly available at https://github.com/naver/anny-fit.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。