arXiv:2508.07112cs.CVcs.LG2025-08

用6维几何描述符提升2D到3D姿态估计的跨域泛化能力

AugLift: Depth-Aware Input Reparameterization Improves Domain Generalization in 2D-to-3D Pose Lifting

  • 将2D坐标转为包含深度置信度的6维几何描述符
  • 跨数据集平均降低10.1%的3D误差,对遮挡关节效果显著
  • 无需额外设备,可兼容任意模型,适合做域泛化研究

基于提升的3D人体姿态估计从2D关键点推断3D关节,但泛化能力差,因仅依赖$(x,y)$坐标会丢失几何信息。本文提出AugLift,通过两个模块将输入表示从2D坐标改为6维几何描述符:(1)不确定性感知深度描述符(UADD)——从现成单目深度图中提取的紧凑元组$(c, d, d_{\min}, d_{\max})$;(2)尺度归一化组件,处理训练与测试距离差异。AugLift无需新传感器、新数据或架构改动,仅扩展输入层,可在表示层面与任何提升架构或域泛化方法组合。在检测设置下,跨四个数据集、四种架构,平均降低10.1%的跨域MPJPE,且分布内精度提升4.0%;后验分析显示收益集中在新姿态和遮挡关节。在真值2D设置下,结合PoseAug的可微域泛化,在3DHP(62.4mm)、3DPW(92.6mm)上达当前最优,分别优于PoseAug 14.5%和22.2%,证明基础模型深度提供与显式3D增强互补的真实几何信号。代码将公开。

原文摘要 · Abstract (English)

Lifting-based 3D human pose estimation infers 3D joints from 2D keypoints but generalizes poorly because $(x,y)$ coordinates alone are an ill-posed, sparse representation that discards geometric information modern foundation models can recover. We propose \emph{AugLift}, which changes the representation format of lifting from 2D coordinates to a 6D geometric descriptor via two modules: (1) an \emph{Uncertainty-Aware Depth Descriptor} (UADD) -- a compact tuple $(c, d, d_{\min}, d_{\max})$ extracted from a confidence-scaled neighborhood of an off-the-shelf monocular depth map -- and (2) a scale normalization component that handles train/test distance shifts. AugLift requires no new sensors, no new data collection, and no architectural changes beyond widening the input layer; because it operates at the representation level, it is composable with any lifting architecture or domain generalization technique. In the detection setting, AugLift reduces cross-dataset MPJPE by $10.1$% on average across four datasets and four lifting architectures while improving in-distribution accuracy by $4.0$%; post-hoc analysis shows gains concentrate on novel poses and occluded joints. In the ground-truth 2D setting, combining AugLift with PoseAug's differentiable domain generalization achieves state-of-the-art cross-dataset performance ($62.4$\,mm on 3DHP, $92.6$\,mm on 3DPW; $14.5$% and $22.2$% over PoseAug), demonstrating that foundation-model depth provides genuine geometric signal complementary to explicit 3D augmentation. Code will be made publicly available.

3D姿态估计域泛化深度感知表示学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。