用生成对抗信号消除干扰,让毫米波人体姿态估计跨人跨环境更准
GenHPE: Generative Counterfactuals for 3D Human Pose Estimation with Radio Frequency Signals
- 通过操控骨骼标签生成反事实射频信号,分离身体动作与干扰因素
- 跨主体/跨环境姿态估计误差降低52.2mm和10.6mm,显著优于现有方法
- 适合做无摄像头、抗遮挡的人体姿态感知系统,尤其在复杂环境
人体姿态估计(HPE)通过检测人体关节位置服务于多种场景。相比摄像头,基于射频(RF)信号的HPE具有非侵入性且对恶劣环境更具鲁棒性,利用人体移动引起的信号变化。然而,现有研究局限于特定域的HPE,受域内混杂因子影响,难以泛化至新域,导致性能下降。具体而言,不同身体部位引起的信号变化相互纠缠,包含个体特异性混杂因子;同时,射频信号还受环境噪声干扰,存在环境特异性混杂因子。本文提出GenHPE,一种3D HPE方法,通过生成反事实射频信号以消除域特异性混杂因子。GenHPE训练生成模型,条件于人体骨骼标签,学习身体部位与混杂因子如何影响射频信号。通过操纵骨骼标签(如移除身体部位)作为反事实条件,生成反事实射频信号。反事实信号间的差异近似消除域特异性混杂因子,并正则化编码器-解码器模型,使其学习域无关表征,从而实现跨域3D HPE的泛化。我们在三个公开数据集(来自WiFi、超宽带和毫米波)上评估GenHPE。实验结果表明,其优于现有最先进方法,跨主体HPE误差减少最多达52.2mm,跨环境HPE误差减少10.6mm。
原文摘要 · Abstract (English)
Human pose estimation (HPE) detects the positions of human body joints for various applications. Compared to using cameras, HPE using radio frequency (RF) signals is non-intrusive and more robust to adverse conditions, exploiting the signal variations caused by human interference. However, existing studies focus on single-domain HPE confined by domain-specific confounders, which cannot generalize to new domains and result in diminished HPE performance. Specifically, the signal variations caused by different human body parts are entangled, containing subject-specific confounders. RF signals are also intertwined with environmental noise, involving environment-specific confounders. In this paper, we propose GenHPE, a 3D HPE approach that generates counterfactual RF signals to eliminate domain-specific confounders. GenHPE trains generative models conditioned on human skeleton labels, learning how human body parts and confounders interfere with RF signals. We manipulate skeleton labels (i.e., removing body parts) as counterfactual conditions for generative models to synthesize counterfactual RF signals. The differences between counterfactual signals approximately eliminate domain-specific confounders and regularize an encoder-decoder model to learn domain-independent representations. Such representations help GenHPE generalize to new subjects/environments for cross-domain 3D HPE. We evaluate GenHPE on three public datasets from WiFi, ultra-wideband, and millimeter wave. Experimental results show that GenHPE outperforms state-of-the-art methods and reduces estimation errors by up to 52.2mm for cross-subject HPE and 10.6mm for cross-environment HPE.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。