用多模态自监督预训练提升IMU动作识别，小样本下准确率最高提升15%。
PRIMUS: Pretraining IMU Encoders with Multimodal Self-Supervision
- 设计新预训练目标，融合自监督与多模态近邻监督
- 仅需每类少于500个标注样本，准确率最高提升15%
- 适用于跨域任务，适合健康监测等小样本场景
通过嵌入个人设备的惯性测量单元（IMU）感知人体运动，已在健康与保健领域催生重要应用。然而，标注的IMU数据稀缺，而未标注或弱标注数据丰富。对于视频或文本模态，常采用“预训练+微调”策略，利用大量弱标注数据构建强特征提取器，再以少量标注数据适配具体任务。但针对IMU数据的预训练方法尚不明确，且缺乏对跨域任务的评估。本文提出PRIMUS：一种基于新颖预训练目标的IMU编码器预训练方法，该目标经下游任务表现验证，在同域与跨域数据集上均有效。其结合自监督、多模态与最近邻监督，显著提升下游性能。在每类样本少于500个时，相比现有最优基线，测试准确率最高提升15%。为促进社区发展，代码已开源至github.com/nokia-bell-labs/pretrained-imu-encoders。
原文摘要 · Abstract (English)
Sensing human motions through Inertial Measurement Units (IMUs) embedded in personal devices has enabled significant applications in health and wellness. Labeled IMU data is scarce, however, unlabeled or weakly labeled IMU data can be used to model human motions. For video or text modalities, the "pretrain and adapt" approach utilizes large volumes of unlabeled or weakly labeled data to build a strong feature extractor, followed by adaptation to specific tasks using limited labeled data. However, pretraining methods are poorly understood for IMU data, and pipelines are rarely evaluated on out-of-domain tasks. We propose PRIMUS: a method for PRetraining IMU encoderS that uses a novel pretraining objective that is empirically validated based on downstream performance on both in-domain and out-of-domain datasets. The PRIMUS objective effectively enhances downstream performance by combining self-supervision, multimodal, and nearest-neighbor supervision. With fewer than 500 labeled samples per class, PRIMUS improves test accuracy by up to 15%, compared to state-of-the-art baselines. To benefit the broader community, we have open-sourced our code at github.com/nokia-bell-labs/pretrained-imu-encoders.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。