arXiv:2507.15349cs.LGcs.AI2025-07

FLock实现70B大模型去中心化安全微调,防中毒攻击且提升跨域泛化。

Scaling Decentralized Learning with FLock

  • 用区块链+经济激励替代中心服务器,构建可审计的去中心协作协议。
  • 在70B模型上验证抗后门攻击,成功率降低超68%,跨域性能优于孤立训练。
  • 适合关注大模型安全协同、去中心化训练的研究者与开发者。

微调大语言模型(LLMs)受限于去中心化方案中缺乏集中控制及巨大的计算通信开销。传统联邦学习(FL)虽保护数据隐私,但依赖中心服务器,易成单点攻击漏洞并面临投毒攻击风险。将该方向扩展至70B参数模型,在异构、不可信环境中仍存在巨大瓶颈。本文提出FLoC,一种用于安全高效协同微调大模型的去中心化框架。通过集成基于区块链的信任层与经济激励机制,FLoC以安全可审计协议取代中心聚合器,实现互不信任方间的协作。我们首次在安全、多领域去中心化环境下实证验证了70B模型的微调。实验表明,FLoC框架能有效防御破坏标准FL优化器的后门投毒攻击,并促进知识协同迁移。结果模型的对抗攻击成功率达68%以上降低,全局模型还展现出优于独立训练专用数据模型的跨域泛化能力。

原文摘要 · Abstract (English)

Fine-tuning the large language models (LLMs) are prevented by the deficiency of centralized control and the massive computing and communication overhead on the decentralized schemes. While the typical standard federated learning (FL) supports data privacy, the central server requirement creates a single point of attack and vulnerability to poisoning attacks. Generalizing the result in this direction to 70B-parameter models in the heterogeneous, trustless environments has turned out to be a huge, yet unbroken bottleneck. This paper introduces FLock, a decentralized framework for secure and efficient collaborative LLM fine-tuning. Integrating a blockchain-based trust layer with economic incentives, FLock replaces the central aggregator with a secure, auditable protocol for cooperation among untrusted parties. We present the first empirical validation of fine-tuning a 70B LLM in a secure, multi-domain, decentralized setting. Our experiments show the FLock framework defends against backdoor poisoning attacks that compromise standard FL optimizers and fosters synergistic knowledge transfer. The resulting models show a >68% reduction in adversarial attack success rates. The global model also demonstrates superior cross-domain generalization, outperforming models trained in isolation on their own specialized data.

去中心化大模型微调安全协作区块链

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。