在智能家居中用4比特量化模型实现私密的联邦指令微调。
Edge-FIT: Federated Instruction Tuning of Quantized LLMs for Privacy-Preserving Smart Home Environments
- 边缘设备上结合4比特量化与低秩适配，降低通信和算力开销。
- 在家庭网关上微调70亿参数模型，准确率达F1=0.89。
- 支持小模型如Phi-3-mini，适合资源受限的隐私保护场景。
本文提出Edge-FIT（边缘联邦指令微调）框架，用于在边缘端对大语言模型进行可扩展的联邦指令微调。传统联邦学习方法（如FedAvg）在面对大语言模型的海量参数时面临挑战。本方案将联邦学习与4比特量化低秩适配（QLORA）结合，有效缓解了通信与计算开销问题。通过筛选通用数据集Databricks Dolly 15k以适配物联网领域，实验表明，在家庭网关部署的经微调的Llama 2(7B)模型达到F1分数0.89。同时，使用3.8亿参数的Phi-3-mini模型也验证了该框架在资源受限场景下的可行性，证明其适用于分布式智能家庭环境中的大模型部署。
原文摘要 · Abstract (English)
This paper proposes Edge-FIT (Federated Instruction Tuning on the Edge), a scalable framework for Federated Instruction Tuning (FIT) of Large Language Models (LLMs). Traditional Federated Learning (TFL) methods, like FedAvg, fail when confronted with the massive parameter size of LLMs [3], [6]. Our Edge-FIT framework combines federated learning with 4-bit Quantized Low-Rank Adaptation (QLORA), mitigating the core issues of communication and computational overhead. We demonstrate this by filtering the general-purpose Databricks Dolly 15k dataset for the IoT domain. Experimental results show the Edge-FIT tuned Llama 2(7B) achieves an F1-Score of 0.89. We also demonstrate a viable trade-off using the 3.8B Phi-3-mini model, validating Edge-FIT as a scalable framework for decentralized LLM deployment on home compute gateways.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。