用自监督学习提升WiFi信号对人体姿态的精准识别能力
WiFi-JEPA: Self-supervised Learning for WiFi-CSI 3D Human Pose Estimation

- 通过掩码潜空间预测,避免硬件噪声干扰,学习原生无线信号特征
- 在真实与模拟数据上均达到当前最佳性能,单人多人姿态估计全胜
- 适合隐私敏感场景,无需摄像头标注,可扩展至大规模部署
WiFi信道状态信息(CSI)可在无摄像头环境下实现隐私保护的人体姿态感知,但现有方法在环境变化下表现不佳,且依赖昂贵的摄像头标注流程。本文提出WiFi-JEPA,一种自监督框架,通过预测被掩码的潜空间嵌入来学习原生CSI表示,而非重建含硬件特异性噪声的原始信号。该方法有三方面贡献:(i) 针对通道、时间、链路(C,T,L)张量设计了特定标记化与链路掩码策略,强制模型从其他空间链路预测单一视图,捕捉跨链路相关性以反映3D空间结构;(ii) 构建基于射线追踪的CSI仿真管道,利用随机几何体生成多样化无标注数据,实现可扩展预训练;(iii) 在Person-in-WiFi-3D数据集上,无论单人还是多人3D姿态估计,均超越已有基线。同时证明模拟数据能有效补充真实数据,而四种视觉原生自监督目标性能低于从零训练,反观WiFi-JEPA始终提升下游任务表现。
原文摘要 · Abstract (English)
WiFi Channel State Information (CSI) enables privacy-preserving human pose sensing in camera-denied environments, but existing WiFi-based pose estimators often fail under environment shifts and rely on costly camera-based annotation pipelines that limit scale. We propose WiFi-JEPA, a self-supervised framework that learns CSI-native representations by predicting masked latent embeddings instead of reconstructing raw CSI signals that may contain hardware-specific artifacts. WiFi-JEPA makes three contributions: (i) CSI-specific tokenization and link masking tailored to the CSI tensor over channel, time, and link (C,T,L); masking entire Tx-Rx antenna links forces the model to predict one spatial link view from others, capturing cross-link correlations informative of 3D spatial structure. (ii) A ray-tracing CSI simulation pipeline that generates diverse unlabeled CSI from randomized geometric primitives, providing scalable pre-training data without pose annotations. (iii) State-of-the-art results on Person-in-WiFi-3D: WiFi-JEPA outperforms prior WiFi-CSI baselines on both single- and multi-person 3D pose estimation under the same evaluation protocol. We also show that simulated CSI provides complementary pre-training signal to real CSI, and that four vision-native SSL objectives degrade performance below training from scratch, whereas WiFi-JEPA consistently improves downstream pose estimation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。