用大模型教小模型，让小型模型在家庭活动识别上接近大模型表现。
Knowledge Distillation for LLM-Based Human Activity Recognition in Homes
- 用大模型生成推理数据，指导小模型学习活动识别。
- 小模型参数减少50倍，性能损失小于1%。
- 适合资源受限场景下的智能家庭应用。
人体活动识别（HAR）是情境感知应用的核心问题，尤其在智能家居和辅助生活领域。近期研究表明，大型语言模型（LLMs）可用于家庭环境中的HAR任务，达到高精度并解决关键挑战。本文在两个最先进的数据集上提供了新实验结果，展示了不同规模的LLM在HAR任务上的性能变化。我们进一步探索了知识蒸馏技术，利用大模型生成的活动识别推理样例来微调小型LLM。实验表明，经过蒸馏训练的小模型在性能上几乎与最大规模的LLM相当，但参数量仅为后者的1/50。
原文摘要 · Abstract (English)
Human Activity Recognition (HAR) is a central problem for context-aware applications, especially for smart homes and assisted living. A few very recent studies have shown that Large Language Models (LLMs) can be used for HAR at home, reaching high performance and addressing key challenges. In this paper, we provide new experimental results regarding the use of LLMs for HAR, on two state-of-the-art datasets. More specifically, we show how recognition performance evolves depending on the size of the LLM used. Moreover, we experiment on the use of knowledge distillation techniques to fine-tune smaller LLMs with HAR reasoning examples generated by larger LLMs. We show that such fine-tuned models can perform almost as well as the largest LLMs, while having 50 times less parameters.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。