用自监督学习提升雷达信号多人姿态估计精度,不依赖图像且更高效。
Self-supervised One-Stage Learning for RF-based Multi-Person Pose Estimation
- 直接处理原始雷达信号,分组嵌入后用多头注意力融合。
- 在[email protected]上比以往方法提升最多15个百分点。
- 新自监督机制适合遮挡或新位置场景,人越多越明显。
在多人姿态估计(MPPE)领域,基于射频(RF)的方法可在光照不足或视线受阻时稳定工作。现有方法通常分为两类:一是通过复杂预处理将RF信号转换为热图图像,虽性能良好但计算开销大;二是直接对原始信号使用深度嵌入网络,虽预处理简单但精度与泛化能力较差。本文提出一种轻量级单阶段模型,直接处理原始RF信号。通过子组划分信号,并用共享单层CNN结合多头注意力进行嵌入,显著优于全信号一次性输入的深层复杂网络。此外,提出一种新自监督学习(SSL)方法:利用未遮挡子组和其余遮挡子组输入,预测遮挡部分的潜在表示。实验表明,该模型在[email protected]指标上相比先前纯信号方法最高提升15。尤其在新位置或障碍物遮挡场景下,随着人数增加,该自监督方法带来的性能增益更显著。代码与数据集已开源于GitHub:https://github.com/sshnan7/SOSPE。
原文摘要 · Abstract (English)
In the field of Multi-Person Pose Estimation (MPPE), Radio Frequency (RF)-based methods can operate effectively regardless of lighting conditions and obscured line-of-sight situations. Existing RF-based MPPE methods typically involve either 1) converting RF signals into heatmap images through complex preprocessing, or 2) applying a deep embedding network directly to raw RF signals. The first approach, while delivering decent performance, is computationally intensive and time-consuming. The second method, though simpler in preprocessing, results in lower MPPE accuracy and generalization performance. This paper proposes an efficient and lightweight one-stage MPPE model based on raw RF signals. By sub-grouping RF signals and embedding them using a shared single-layer CNN followed by multi-head attention, this model outperforms previous methods that embed all signals at once through a large and deep CNN. Additionally, we propose a new self-supervised learning (SSL) method that takes inputs from both one unmasked subgroup and the remaining masked subgroups to predict the latent representations of the masked data. Empirical results demonstrate that our model improves MPPE accuracy by up to 15 in [email protected] compared to previous methods using raw RF signals. Especially, the proposed SSL method has shown to significantly enhance performance improvements when placed in new locations or in front of obstacles at RF antennas, contributing to greater performance gains as the number of people increases. Our code and dataset is open at Github. https://github.com/sshnan7/SOSPE .
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。