聚焦联邦微调LLM的可访问性，梳理黑箱模式下的高效训练方法。
On the Evolution of Federated Post-Training Large Language Models: A Model Accessibility View
- 按模型访问权限分白/灰/黑盒三类，构建系统分类体系
- 提出将LLM视为黑箱API的联邦微调新范式
- 适合关注隐私保护与低资源部署的研究者
联邦学习（FL）可在保护客户端数据隐私的前提下，实现跨分散数据源的模型训练。近期研究致力于在联邦框架下高效微调大语言模型（LLMs），以应对计算与通信挑战。然而，现有方法多依赖对模型内部信息的访问，这在真实场景中常受限制。为此，仅依赖推理接口的黑箱联邦微调（black-box FedLLM）范式应运而生。本文系统综述了联邦微调大语言模型的研究进展，提出一个双维度分类体系：基于模型访问方式与参数效率优化。将FedLLM方法划分为白盒、灰盒与黑盒三类，并归纳各类型代表性方法。同时，综述将大语言模型视为黑箱推理API的新兴研究，探讨未来潜在方向与关键挑战。
原文摘要 · Abstract (English)
Federated Learning (FL) enables training models across decentralized data silos while preserving client data privacy. Recent research has explored efficient methods for post-training large language models (LLMs) within FL to address computational and communication challenges. While existing approaches often rely on access to LLMs' internal information, which is frequently restricted in real-world scenarios, an inference-only paradigm (black-box FedLLM) has emerged to address these limitations. This paper presents a comprehensive survey on federated tuning for LLMs. We propose a taxonomy categorizing existing studies along two axes: model access-based and parameter efficiency-based optimization. We classify FedLLM approaches into white-box, gray-box, and black-box techniques, highlighting representative methods within each category. We review emerging research treating LLMs as black-box inference APIs and discuss promising directions and open challenges for future research.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。