用预训练大模型做传感器动作识别,省数据又快又准。
Efficient and Adaptive Human Activity Recognition via LLM Backbones

- 用大语言模型作时间特征提取主干,不从头训练
- 低数据量下快速收敛,跨数据集迁移能力强
- 适合资源受限、需快速适应新场景的智能设备
人体动作识别(HAR)是普适计算系统的核心任务,要求模型在严苛计算约束下仍能应对多样且动态变化的部署环境。近年基于Transformer的方法显著提升了识别性能,但通常依赖特定任务模型从零训练,导致训练成本高、数据需求大,且对领域偏移适应性差。本文提出范式转变:复用预训练大语言模型(LLM)作为通用时序主干处理传感器数据,而非设计专用Transformer。为弥合惯性信号与语言模型间的模态差异,引入结构化卷积投影,将多变量加速度计与陀螺仪信号映射至LLM隐空间。预训练主干保持冻结,通过参数高效微调方法LoRA进行适配,大幅降低可训练参数量和整体训练开销。在标准HAR基准上的大量实验表明,该方法实现快速收敛、强数据效率及鲁棒的跨数据集迁移能力,尤其在低数据和少样本设置中表现突出。同时结果揭示卷积前端与LLM的互补作用:局部不变性由信号层处理,长程时序依赖由预训练主干捕捉。总体而言,本工作证明了LLM可作为自适应HAR系统的实用、经济、可扩展基础,拓展了基础模型在原始语言领域外的复用路径。
原文摘要 · Abstract (English)
Human Activity Recognition (HAR) is a core task in pervasive computing systems, where models must operate under strict computational constraints while remaining robust to heterogeneous and evolving deployment conditions. Recent advances based on Transformer architectures have significantly improved recognition performance, but typically rely on task-specific models trained from scratch, resulting in high training cost, large data requirements, and limited adaptability to domain shifts. In this paper, we propose a paradigm shift that reuses large pretrained language models (LLMs) as generic temporal backbones for sensor-based HAR, instead of designing domain-specific Transformers. To bridge the modality gap between inertial time series and language models, we introduce a structured convolutional projection that maps multivariate accelerometer and gyroscope signals into the latent space of the LLM. The pretrained backbone is kept frozen and adapted using parameter-efficient Low-Rank Adaptation (LoRA), drastically reducing the number of trainable parameters and the overall training cost. Through extensive experiments on standard HAR benchmarks, we show that this approach enables rapid convergence, strong data efficiency, and robust cross-dataset transfer, particularly in low-data and few-shot settings. At the same time, our results highlight the complementary roles of convolutional frontends and LLMs, where local invariances are handled at the signal level while long-range temporal dependencies are captured by the pretrained backbone. Overall, this work demonstrates that LLMs can serve as a practical, frugal, and scalable foundation for adaptive HAR systems, opening new directions for reusing foundation models beyond their original language domain.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。