轻量级框架支持私有数据下大模型安全微调
VFLAIR-LLM: A Comprehensive Framework and Benchmark for Split Learning of LLMs
- 提出两种模型分割方案，适配资源受限环境
- 覆盖18个数据集，验证5种攻击与9种防御策略
- 适合关注隐私保护与边缘部署的研究者
随着大语言模型（LLMs）的发展，其应用已扩展至众多领域。然而，数据隐私顾虑使用户难以直接使用 LLM API，而本地部署又带来高昂计算成本，导致在资源受限条件下实现安全的 LLM 适应面临挑战。为此，协作学习方法如分拆学习（Split Learning, SL）提供了资源高效且隐私保护的解决方案。本文提出 VFLAIR-LLM（https://github.com/FLAIR-THU/VFLAIR-LLM），一个可扩展、轻量化的分拆学习框架，支持大模型在资源受限环境中的隐私保护推理与微调。该框架提供两种模型分割方式，支持三种任务类型和18个数据集，并集成标准模块用于实现和评估攻击与防御方法。我们在多种分拆学习大模型（SL-LLM）设置下，基准测试了5种攻击与9种防御策略，为实际应用中模型分割配置、防御策略及超参数选择提供了具体洞见与建议。
原文摘要 · Abstract (English)
With the advancement of Large Language Models (LLMs), LLM applications have expanded into a growing number of fields. However, users with data privacy concerns face limitations in directly utilizing LLM APIs, while private deployments incur significant computational demands. This creates a substantial challenge in achieving secure LLM adaptation under constrained local resources. To address this issue, collaborative learning methods, such as Split Learning (SL), offer a resource-efficient and privacy-preserving solution for adapting LLMs to private domains. In this study, we introduce VFLAIR-LLM (available at https://github.com/FLAIR-THU/VFLAIR-LLM), an extensible and lightweight split learning framework for LLMs, enabling privacy-preserving LLM inference and fine-tuning in resource-constrained environments. Our library provides two LLM partition settings, supporting three task types and 18 datasets. In addition, we provide standard modules for implementing and evaluating attacks and defenses. We benchmark 5 attacks and 9 defenses under various Split Learning for LLM(SL-LLM) settings, offering concrete insights and recommendations on the choice of model partition configurations, defense strategies, and relevant hyperparameters for real-world applications.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。