用雷达实现低成本3D人体姿态估计，仅需2D关键点和3D包围框标签。
RAPTR: Radar-based 3D Pose Estimation using Transformer
- 分两阶段解码，结合伪3D可变形注意力融合多视角雷达特征。
- 在HIBER和MMVR数据集上误差分别降低34.3%和76.9%。
- 适合缺乏精细标注的复杂室内场景，标注成本大幅降低。
基于雷达的室内3D人体姿态估计通常依赖于细粒度的3D关键点标注，这在存在杂乱、遮挡或多人体的复杂室内环境中代价高昂。本文提出弱监督下的RAPTR（RAdar Pose esTimation using tRansformer），仅需3D边界框和2D关键点标签，采集更简便且更具可扩展性。RAPTR采用双阶段姿态解码器架构，引入伪3D可变形注意力，将多视角雷达特征增强姿态查询：姿态解码器利用3D边界框标签与3D模板损失估计初始3D姿态，缓解深度模糊；关节解码器则结合2D关键点标签与3D重力损失进行精修。在两个室内雷达数据集上评估，RAPTR优于现有方法，在HIBER上关节位置误差降低34.3%，在MMVR上降低76.9%。代码已开源。
原文摘要 · Abstract (English)
Radar-based indoor 3D human pose estimation typically relied on fine-grained 3D keypoint labels, which are costly to obtain especially in complex indoor settings involving clutter, occlusions, or multiple people. In this paper, we propose \textbf{RAPTR} (RAdar Pose esTimation using tRansformer) under weak supervision, using only 3D BBox and 2D keypoint labels which are considerably easier and more scalable to collect. Our RAPTR is characterized by a two-stage pose decoder architecture with a pseudo-3D deformable attention to enhance (pose/joint) queries with multi-view radar features: a pose decoder estimates initial 3D poses with a 3D template loss designed to utilize the 3D BBox labels and mitigate depth ambiguities; and a joint decoder refines the initial poses with 2D keypoint labels and a 3D gravity loss. Evaluated on two indoor radar datasets, RAPTR outperforms existing methods, reducing joint position error by $34.3\%$ on HIBER and $76.9\%$ on MMVR. Our implementation is available at https://github.com/merlresearch/radar-pose-transformer.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。