让可穿戴设备在任意佩戴方式下都能准确识别人体动作
AnyMo: Geometry-Aware Setup-Agnostic Modeling of Human Motion in the Wild

- 用物理仿真生成多位置传感器数据,解决佩戴方式差异问题
- 跨14个数据集零样本识别准确率提升11.7%,检索效果提高超28%
- 适合做通用可穿戴动作理解,尤其适合无标签场景
随着可穿戴设备日益融入日常生活,它们为野外持续感知人体运动提供了可能。但惯性信号高度依赖于传感设置,包括身体位置、安装位置、传感器方向、设备硬件和采样协议。这种设置依赖性使得运动表征难以跨设备与数据集迁移,限制了可穿戴IMU在封闭集识别之外的广泛应用。我们提出AnyMo,一种几何感知的设置无关人体运动建模框架。AnyMo通过在密集身体表面位置上进行物理基础的IMU仿真,生成多样且合理的合成信号;从成对的合成位置视图和掩码部分观测中预训练图编码器;将多位置IMU数据分词为全身运动标记;并通过与大语言模型对齐实现运动-语言理解。我们在三个互补任务上评估AnyMo:跨14个未见下游数据集的零样本活动识别、跨模态检索和可穿戴IMU动作描述生成,结果表明其在HAR任务上平均准确率/精确率/F1值提升11.7%/11.6%/22.6%,零样本IMU-to-text和text-to-IMU检索的MRR分别提升15.9%和28.6%,零样本描述生成的BERT-F1提升18.8%。这些结果验证了AnyMo作为野外可穿戴运动理解通用模型的有效性。
原文摘要 · Abstract (English)
As wearable and mobile devices become increasingly embedded in daily life, they offer a practical way to continuously sense human motion in the wild. But inertial signals are highly dependent on the sensing setup, including body location, mounting position, sensor orientation, device hardware, and sampling protocol. This setup dependence makes it difficult to learn motion representations that transfer across devices and datasets, and limits the broader use of wearable IMUs beyond closed-set recognition. We introduce AnyMo, a geometry-aware framework for setup-agnostic human motion modeling. AnyMo uses physics-grounded IMU simulation over dense body-surface placements to generate diverse and plausible synthetic signals, pre-trains a graph encoder from paired synthetic placement views and masked partial observations, tokenizes multi-position IMU into full-body motion tokens, and aligns these tokens with an LLM for motion-language understanding. We evaluate AnyMo on three complementary tasks: zero-shot activity recognition across 14 unseen downstream datasets, cross-modal retrieval, and wearable IMU motion captioning, where it improves average Accuracy/F1/R@2 by 11.7\%/11.6\%/22.6\% on HAR, increases zero-shot IMU-to-text and text-to-IMU retrieval MRR by 15.9\% and 28.6\%, respectively, and improves zero-shot captioning BERT-F1 by 18.8\%. These results support AnyMo as a generalist model for wearable motion understanding in the wild. Project page: https://baiyuchen.com/project/AnyMo.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。