在保护隐私的前提下，用极少数据高效训练大模型指令响应能力。
Federated Data-Efficient Instruction Tuning for Large Language Models
- 从客户端选取代表性数据子集进行联邦微调，减少冗余。
- 仅用不到1.5%的数据，使新任务的ROUGE-L提升10.72%。
- 适合数据敏感场景，如医疗、金融等私有数据应用。
指令微调是提升预训练大语言模型对人类指令响应能力的关键步骤。联邦学习（FL）能利用客户端的海量私有指令数据，通过增加数据多样性成为大模型微调的热门方法。然而现有联邦微调方法直接使用全部本地数据，导致计算开销过大且易过拟合；而集中式高效方案因隐私问题不适用于联邦场景。本文提出FedHDS，一种联邦数据高效的指令微调方法，仅使用边缘侧数据的代表性子集进行模型训练，有效降低客户端内与跨客户端的数据冗余，且无需共享原始数据。在多种大模型、数据集和数据划分方式下的实验表明，相比当前最优全数据联邦指令微调方法，FedHDS在未见过的任务上平均提升ROUGE-L达10.72%，同时仅使用不足1.5%的数据样本，训练效率最高提升数十倍。
原文摘要 · Abstract (English)
Instruction tuning is a crucial step in improving the responsiveness of pretrained large language models (LLMs) to human instructions. Federated learning (FL) helps to exploit the use of vast private instruction data from clients, becoming popular for LLM tuning by improving data diversity. Existing federated tuning simply consumes all local data, causing excessive computational overhead and overfitting to local data, while centralized data-efficient solutions are not suitable for FL due to privacy concerns. This work presents FedHDS, a federated data-efficient instruction tuning approach, which tunes LLMs with a representative subset of edge-side data. It reduces the data redundancy at both intra- and inter-client levels without sharing raw data. Experiments with various LLMs, datasets and partitions show that FedHDS improves Rouge-L on unseen tasks by an average of 10.72% over the SOTA full-data federated instruction tuning methods, while using less than 1.5% of the data samples, improving training efficiency by up to tens of times.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。