用无线信号生成动作描述，保护隐私还能准确识别人体动作。
WiFi2Cap: Semantic Action Captioning from Wi-Fi CSI via Limb-Level Semantic Alignment
- 通过视觉语言教师引导，将无线信号对齐到语义空间。
- 引入镜像一致性损失，解决左右方向混淆问题。
- 首个支持自然语言生成的无线信号动作数据集，适合隐私敏感场景。
隐私保护的室内人体活动语义理解至关重要，但现有基于无线信道状态信息（Wi-Fi CSI）的系统多聚焦于姿态估计或预定义动作分类，而非细粒度语言生成。将CSI映射为自然语言描述仍具挑战，源于无线信号与语言间的语义鸿沟，以及左/右肢体方向敏感的歧义问题。本文提出WiFi2Cap，一个三阶段框架，可直接从Wi-Fi CSI生成动作描述。其视觉-语言教师模型从同步视频-文本对中学习可迁移的监督信号，而CSI学生模型被对齐至教师的视觉空间与文本嵌入。为改善方向敏感的描述准确性，提出镜像一致性损失，减少镜像动作及左右混淆。随后，前缀调优的语言模型从CSI嵌入生成动作描述。此外，构建了首个同步的CSI-RGB-句子基准数据集WiFi2Cap Dataset。实验表明，WiFi2Cap在BLEU-4、METEOR、ROUGE-L、CIDEr和SPICE指标上持续优于基线方法，验证了其在隐私友好语义感知中的有效性。
原文摘要 · Abstract (English)
Privacy-preserving semantic understanding of human activities is important for indoor sensing, yet existing Wi-Fi CSI-based systems mainly focus on pose estimation or predefined action classification rather than fine-grained language generation. Mapping CSI to natural-language descriptions remains challenging because of the semantic gap between wireless signals and language and direction-sensitive ambiguities such as left/right limb confusion. We propose WiFi2Cap, a three-stage framework for generating action captions directly from Wi-Fi CSI. A vision-language teacher learns transferable supervision from synchronized video-text pairs, and a CSI student is aligned to the teacher's visual space and text embeddings. To improve direction-sensitive captioning, we introduce a Mirror-Consistency Loss that reduces mirrored-action and left-right ambiguities during cross-modal alignment. A prefix-tuned language model then generates action descriptions from CSI embeddings. We also introduce the WiFi2Cap Dataset, a synchronized CSI-RGB-sentence benchmark for semantic captioning from Wi-Fi signals. Experimental results show that WiFi2Cap consistently outperforms baseline methods on BLEU-4, METEOR, ROUGE-L, CIDEr, and SPICE, demonstrating effective privacy-friendly semantic sensing.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。