用检索代替训练,让边缘设备低成本识别人体动作。
RAG-HAR+: Towards Cost-Efficient LLM-Based Human Activity Recognition for Edge Deployment

- 先用检索匹配相似动作样本,再用大模型处理模糊情况。
- 在6个数据集上性能持平或更好,大模型调用减少超70%。
- 适合资源受限的可穿戴设备,部署门槛低。
从可穿戴传感器进行人体活动识别(HAR)支持医疗、康复、健身追踪和智能环境应用。现有深度学习方法需特定数据集训练、大量标注数据,并反复适应新传感器设置或活动分类体系。检索增强生成人体活动识别(RAG-HAR)通过将HAR视为无需训练、基于检索的任务,利用传感器窗口的统计描述检索相似标注样本,引导大模型分类。本文提出RAG-HAR+,一种以检索为主、成本优化的扩展方案:离线使用检索设计代理从多样化运动描述池中构建数据集特异性特征组,使传感器窗口能基于更契合活动模式的特征进行比对;推理时,对检索证据强的样本采用多数投票,仅将不确定案例交由大模型模糊性解析代理处理。在六个HAR基准测试中,RAG-HAR+保持竞争力或提升性能,同时显著降低大模型使用量、令牌消耗和推理时间。进一步扩展RAG-HAR移动端原型,验证了检索优先、大模型辅助的HAR在移动传感场景中的实际可行性。
原文摘要 · Abstract (English)
Human Activity Recognition (HAR) from wearable sensors supports applications in healthcare, rehabilitation, fitness tracking, and smart environments. Yet, existing deep learning approaches require dataset-specific training, large labeled corpora, and repeated adaptation to new sensor settings or activity taxonomies. Retrieval-Augmented Generation for Human Activity Recognition (RAG-HAR) addresses this by framing HAR as a training-free, retrieval-augmented task, in which statistical descriptions of sensor windows are used to retrieve similar labeled examples that guide LLM-based classification. We introduce RAG-HAR+, a retrieval-first and cost-optimized extension that strengthens retrieval while reducing dependence on LLM-based inference. RAG-HAR+ uses an offline Retrieval Designer Agent to design dataset-specific feature groups from a diverse pool of motion descriptors, enabling sensor windows to be compared using features better aligned with dataset-specific activity patterns. During inference, RAG-HAR+ uses majority voting over retrieved neighbors for samples with strong retrieval evidence and defers only uncertain cases to an LLM-based Ambiguity Resolver Agent. Across six HAR benchmarks, RAG-HAR+ maintains competitive or improved performance while reducing LLM usage, token consumption, and inference time. We further extend the RAG-HAR mobile prototype to demonstrate the practical feasibility of retrieval-first, LLM-assisted HAR in mobile sensing scenarios.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。