让大模型在设备上全整数训练，省内存省电还保精度。
GSQ-Tuning: Group-Shared Exponents Integer in Fully Quantized Training for LLMs On-Device Fine-tuning
- 用分组共享指数整数格式表示参数，替代浮点运算。
- 相比BF16精度相当，内存减少1.85倍，功耗降5倍。
- 适合资源受限的边缘设备部署大模型微调。
大型语言模型（LLMs）微调技术已取得显著成果。然而，传统微调方法依赖大量浮点计算，在处理敏感数据时引发隐私担忧，且不适用于资源受限的边缘设备。尽管参数高效微调（PEFT）技术减少了可训练参数量，但仍依赖浮点运算，与边缘硬件存在根本性不兼容。本文提出一种新型设备端大模型微调框架——GSQ-Tuning，完全消除推理与训练中的浮点操作。其核心是分组共享指数整数（Group-Shared Exponents Integer）格式，通过参数组间共享指数，高效以整数形式表示模型参数。结合类似LoRA的适配器，实现全整数微调，兼具内存与计算效率。实验表明，该方法在保持与BF16微调相当精度的同时，内存使用降低1.85倍；相比FP8，功耗减少5倍，芯片面积缩小11倍，使大规模模型在边缘设备上的适配成为可能。
原文摘要 · Abstract (English)
Large Language Models (LLMs) fine-tuning technologies have achieved remarkable results. However, traditional LLM fine-tuning approaches face significant challenges: they require large Floating Point (FP) computation, raising privacy concerns when handling sensitive data, and are impractical for resource-constrained edge devices. While Parameter-Efficient Fine-Tuning (PEFT) techniques reduce trainable parameters, their reliance on floating-point arithmetic creates fundamental incompatibilities with edge hardware. In this work, we introduce a novel framework for on-device LLM fine-tuning that eliminates the need for floating-point operations in both inference and training, named GSQ-Tuning. At its core is the Group-Shared Exponents Integer format, which efficiently represents model parameters in integer format using shared exponents among parameter groups. When combined with LoRA-like adapters, this enables fully integer-based fine-tuning that is both memory and compute efficient. We demonstrate that our approach achieves accuracy comparable to BF16-based fine-tuning while significantly reducing 1.85x memory usage. Moreover, compared to FP8, our method can reduce 5x power consumption and 11x chip area with same performance, making large-scale model adaptation feasible on edge devices.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。