HALO模型让可穿戴设备识别动作更通用，能适应不同人体和设备差异。
HALO: A Heterogeneity-Aware Language-Aligned IMU Foundation Model for Open-Set Human Activity Recognition
- 通过自监督学习与语言描述融合，增强对传感器差异的适应能力
- 零样本开放集识别准确率提升13.7个百分点，仅用3500万参数
- 适合跨设备、跨人群的动作识别场景，尤其适合资源受限部署
基于惯性测量单元（IMU）的人体动作识别（HAR）应用广泛，但缺乏能跨个体、设备和动作泛化的统一模型。训练困难源于两个关键挑战：传感异质性（采样率、通道配置、传感器位置差异）以及对未见动作和标签词汇的泛化能力差。本文提出HALO（Heterogeneity-Aware Language-aligned Open-set model），一种面向IMU的领域特定基础模型，采用两阶段训练框架。第一阶段通过异质性感知的自监督学习预训练IMU编码器，包含自适应池化分词、通道独立特征提取和注入自然语言传感器描述的上下文感知传感器条件化。第二阶段通过同义词感知的软对比学习将IMU编码器与文本嵌入对齐，实现无需每数据集分类器的余弦相似度检索式开集识别。在10个公开HAR数据集上训练，7个保留数据集上评估，HALO在全部8个聚合指标上优于5个最先进基线，且在基线输入匹配设置下仍领先4个中的3个。尽管仅有约3500万可训练参数（比最新模型MOMENT少10倍），其零样本开集准确率（覆盖全部87个训练标签）仍提升13.7个百分点。在两个存在严重分布偏移的数据集上，所有模型包括HALO均出现零样本性能崩溃。真实场景视频演示见https://youtu.be/rooVKragtFU
原文摘要 · Abstract (English)
Human Activity Recognition (HAR) using inertial measurement units (IMUs) enables a wide range of applications, yet the field still lacks a unified model that can generalize across diverse subjects, devices, and activities. Training such a model is difficult due to two key challenges: sensing heterogeneity -- differences in sampling rates, channel configurations, and sensor placements -- and poor generalization to unseen activities and label vocabularies. We introduce HALO (Heterogeneity-Aware Language-aligned Open-set model), a domain-specific IMU foundation model that addresses both challenges through a two-stage training framework. Stage 1 pretrains the IMU encoder with heterogeneity-aware self-supervised learning, including adaptive-pooling tokenization, channel-independent feature extraction, and contextualized sensor conditioning that injects natural-language sensor descriptions into each channel embedding. Stage 2 aligns this IMU encoder with text embeddings via synonym-aware soft contrastive learning, enabling open-set recognition via cosine-similarity retrieval without per-dataset classifiers. Trained on 10 public HAR datasets and evaluated on 7 held-out datasets, HALO outperforms five state-of-the-art baselines on all 8 aggregate metrics, and still leads on 3 of 4 settings under baseline-matched inputs. Despite using only ~35M trainable parameters -- 10x fewer than the latest foundation model MOMENT (341.2M) -- HALO improves zero-shot open-set accuracy, measured over all 87 training labels, by 13.7 percentage points. On two further datasets with severe distribution shift, every model including HALO collapses zero-shot. A video demonstration of HALO's performance in real world is available at https://youtu.be/rooVKragtFU
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。