arXiv:2512.17983cs.LGcs.AI2025-12被引 2

用轻量适配器让大模型在小设备上高效识别动作

Parameter-Efficient Fine-Tuning for HAR: Integrating LoRA and QLoRA into Transformer Models

  • 用低秩适配技术只训练少量参数,大幅降低资源消耗
  • 在5个公开数据集上达到全量微调的准确率,参数减少90%以上
  • 适合算力受限的智能穿戴设备,尤其对小样本场景友好

人类活动识别是普适计算中的基础任务。尽管自监督学习和基于Transformer的架构显著提升了识别性能,但在目标设备上将大型预训练模型适配到新领域仍面临计算资源有限的挑战。本文研究了参数高效微调技术,特别是低秩适配(LoRA)和量化低秩适配(QLoRA),作为全模型微调的可扩展替代方案。我们提出一个基于掩码自编码器骨干网络的适配框架,并在五个公开的HAR数据集上采用留一数据集外验证协议进行评估。实验表明,LoRA和QLoRA在保持全量微调识别性能的同时,显著减少了可训练参数数量、内存占用和训练时间。进一步分析显示,LoRA在标注数据有限时仍具备鲁棒性,且适配器秩可灵活控制精度与效率的权衡。QLoRA通过量化冻结权重进一步降低了内存开销,对分类质量影响极小。

原文摘要 · Abstract (English)

Human Activity Recognition is a foundational task in pervasive computing. While recent advances in self-supervised learning and transformer-based architectures have significantly improved HAR performance, adapting large pretrained models to new domains remains a practical challenge due to limited computational resources on target devices. This papers investigates parameter-efficient fine-tuning techniques, specifically Low-Rank Adaptation (LoRA) and Quantized LoRA, as scalable alternatives to full model fine-tuning for HAR. We propose an adaptation framework built upon a Masked Autoencoder backbone and evaluate its performance under a Leave-One-Dataset-Out validation protocol across five open HAR datasets. Our experiments demonstrate that both LoRA and QLoRA can match the recognition performance of full fine-tuning while significantly reducing the number of trainable parameters, memory usage, and training time. Further analyses reveal that LoRA maintains robust performance even under limited supervision and that the adapter rank provides a controllable trade-off between accuracy and efficiency. QLoRA extends these benefits by reducing the memory footprint of frozen weights through quantization, with minimal impact on classification quality.

活动识别轻量微调LoRA边缘计算

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。