用大模型提升传感器动作识别跨数据集性能
Large Language Model-Guided Semantic Alignment for Human Activity Recognition
- 用大模型生成传感器数据的语义解释，对齐不同数据分布
- 跨数据集识别准确率提升12.3%，新动作识别效果显著
- 轻量编码器适合移动端部署，适合医疗与工业场景
基于惯性测量单元（IMU）的人体动作识别（HAR）在医疗、安全和工业生产中至关重要。然而，动作模式、设备类型和传感器位置的差异导致数据集间分布不一致，影响模型性能。为此，我们提出LanHAR系统，利用大语言模型（LLM）生成传感器读数和动作标签的语义解释，实现跨数据集的HAR。该方法通过迭代重生成机制获取高质量语义解释，并采用两阶段训练框架对齐传感器读数与动作标签的语义空间。最终构建的轻量级传感器编码器可将任意传感器读数映射至语义空间，支持移动部署。在五个公开数据集上的实验表明，该方法在跨数据集动作识别和新动作识别任务中均显著优于现有先进方法。代码已开源：https://github.com/DASHLab/LanHAR。
原文摘要 · Abstract (English)
Human Activity Recognition (HAR) using Inertial Measurement Unit (IMU) sensors is critical for applications in healthcare, safety, and industrial production. However, variations in activity patterns, device types, and sensor placements create distribution gaps across datasets, reducing the performance of HAR models. To address this, we propose LanHAR, a novel system that leverages Large Language Models (LLMs) to generate semantic interpretations of sensor readings and activity labels for cross-dataset HAR. This approach not only mitigates cross-dataset heterogeneity but also enhances the recognition of new activities. LanHAR employs an iterative re-generation method to produce high-quality semantic interpretations with LLMs and a two-stage training framework that bridges the semantic interpretations of sensor readings and activity labels. This ultimately leads to a lightweight sensor encoder suitable for mobile deployment, enabling any sensor reading to be mapped into the semantic interpretation space. Experiments on five public datasets demonstrate that our approach significantly outperforms state-of-the-art methods in both cross-dataset HAR and new activity recognition. The source code is publicly available at https://github.com/DASHLab/LanHAR.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。