arXiv:2607.03089cs.LGcs.AI2026-07

用轻量分层编码器把传感器数据转成紧凑标签,让大模型在设备上高效识活动。

STELLA: Efficient Sensor-to-LLM Translation for On-Device Human Activity Recognition

论文配图:STELLA: Efficient Sensor-to-LLM Translation for On-Device Human Activity Recognition
图 1 · 摘自论文原文
  • 将多通道传感器数据压缩为固定长度的紧凑隐向量标签
  • 在七大数据集上提升最多11.83%的识别准确率,设备端个性化再提21.91%
  • 全程本地处理,适合隐私敏感、需自适应的可穿戴设备应用

HAR正越来越多地要求在边缘设备上持续运行,但现有基于大模型的方法难以部署:原始传感器输入过长,云端推理带来延迟和隐私风险,微调后的模型则使通用大模型变为专用分类器。本文提出 STELLA,一种面向设备端 HAR 的高效传感器到大模型转换框架,将负担从大模型适配转移到传感器编码。一个轻量级分层编码器将整个多通道惯性数据窗口压缩为固定数量的紧凑隐向量标签,这些标签被映射到冻结预训练大模型的嵌入空间,并与自然语言提示结合进行标签打分。该方法保留了活动相关的时序与跨通道结构,同时确保大模型计算量在不同传感器配置下保持稳定。STELLA 还支持设备端个性化,仅在少量用户特定标注数据上微调轻量编码器,并引入本地检索上下文增强推理,保证大模型、用户数据与检索内容均保留在设备内。在七个公开 HAR 数据集和八个基准设置中,STELLA 达到新 SOTA 性能,相比先前方法最高提升 11.83% F1;设备端个性化随用户数据积累可再提升 21.91% F1。STELLA 在相同大模型流程下优于代表性时间序列编码器,且在实际移动端与边缘设备预算下实现实时推理,证明高效传感器编码是实现精准、私密、个性化大模型边缘 HAR 的可行路径。

原文摘要 · Abstract (English)

HAR is increasingly expected to run continuously on edge devices, yet recent LLM-based methods remain hard to deploy: raw sensor prompts are long, cloud inference adds latency and privacy risk, and fine-tuned LLM pipelines turn general-purpose models into task-specific classifiers. We present STELLA, an efficient sensor-to-LLM translation framework for on-device HAR that shifts the burden from LLM adaptation to sensor tokenization. A lightweight hierarchical tokenizer compresses an entire multi-channel inertial window into a fixed set of compact latent sensor tokens, which are projected into the embedding space of a frozen pretrained LLM and combined with a natural-language prompt for label scoring. This preserves activity-relevant temporal and cross-channel structure while keeping LLM-side computation predictable across sensor configurations. STELLA also supports on-device personalization, adapting only the lightweight tokenizer on small amounts of user-specific labelled data and augmenting inference with a local retrieval context, keeping the LLM, user data, and retrieval on device. Across seven public HAR datasets and eight benchmark settings, STELLA achieves new state-of-the-art performance, improving over prior methods by up to 11.83% F1; on-device personalization yields up to a further 21.91% F1 as user data accumulates after deployment. STELLA also outperforms representative time-series tokenizers under the same LLM pipeline and achieves real-time inference under practical mobile and edge budgets, showing that efficient sensor tokenization is a practical path toward accurate, private, and personalized LLM-based HAR on edge devices.

边缘计算大模型动作识别传感器编码

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。