arXiv:2410.10624cs.CL2024-10EMNLP被引 58

让大模型读懂传感器数据,实现精准人体行为识别

SensorLLM: Aligning Large Language Models with Motion Sensors for Human Activity Recognition

  • 通过语义对齐让大模型理解传感器时间序列的走势特征
  • 在多个数据集上达到或超过现有最优方法的识别准确率
  • 无需人工标注即可通用跨数据集,适合做智能传感分析

我们提出SensorLLM,一种两阶段框架,使大语言模型(LLMs)能够从传感器时间序列数据中进行人体行为识别(HAR)。尽管大模型具备强大的推理和泛化能力,但因其时间序列缺乏语义上下文、计算开销大以及难以处理数值输入,尚未被充分用于传感器数据。SensorLLM通过传感器-语言对齐阶段,将传感器输入与趋势描述进行匹配,并引入特殊标记标识通道边界,使模型能捕捉数值变化、通道特性和不同长度数据,且无需人工标注。在任务感知微调阶段,模型进一步优化用于HAR分类,性能达到或超越当前最优方法。实验表明,SensorLLM通过人直观的传感器-语言对齐机制,演变为有效的传感器学习者、推理者和分类器,可在多种HAR数据集间良好泛化。该工作为时间序列与文本对齐研究奠定基础,推动传感器数据分析中的基础模型发展。代码已开源:https://github.com/zechenli03/SensorLLM。

原文摘要 · Abstract (English)

We introduce SensorLLM, a two-stage framework that enables Large Language Models (LLMs) to perform human activity recognition (HAR) from sensor time-series data. Despite their strong reasoning and generalization capabilities, LLMs remain underutilized for motion sensor data due to the lack of semantic context in time-series, computational constraints, and challenges in processing numerical inputs. SensorLLM addresses these limitations through a Sensor-Language Alignment stage, where the model aligns sensor inputs with trend descriptions. Special tokens are introduced to mark channel boundaries. This alignment enables LLMs to capture numerical variations, channel-specific features, and data of varying durations, without requiring human annotations. In the subsequent Task-Aware Tuning stage, we refine the model for HAR classification, achieving performance that matches or surpasses state-of-the-art methods. Our results demonstrate that SensorLLM evolves into an effective sensor learner, reasoner, and classifier through human-intuitive Sensor-Language Alignment, generalizing across diverse HAR datasets. We believe this work establishes a foundation for future research on time-series and text alignment, paving the way for foundation models in sensor data analysis. Our codes are available at https://github.com/zechenli03/SensorLLM.

行为识别大模型传感器对齐

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。