arXiv:2503.04971cs.LGcs.AI2025-03中稿 · publication in IEE…被引 3

为边缘多租户联邦学习设计激励机制,提升大模型微调效率与公平性。

Incentivizing Multi-Tenant Split Federated Learning for Foundation Models at the Network Edge

  • 设计价格激励机制,引导设备参与多任务微调。
  • 实测加速大模型微调达3.07倍,满足各类性能目标。
  • 适合边缘计算中需多方协作的大模型应用开发者。

基础模型(FMs)如GPT-4可通过微调在多样下游任务中展现强大生成能力。分割联邦学习(SFL)通过将部分模型计算卸载至边缘服务器,实现资源受限设备上的隐私保护式微调,支持设备与边缘协同优化。实际边缘网络常需服务多个SFL租户以支持多样化任务。然而现有研究多集中于单租户场景,缺乏针对多租户设置的定制化激励机制,难以协调自利设备参与不同任务,确保各租户对模型类型、性能目标和微调截止时间的差异化需求。为此,我们提出新型价格激励机制(PRINCE),引导多个SFL租户提供策略性价格激励,吸引高质量设备参与,实现高效微调。首先,构建抗偏差的全局SFL模型聚合方案,消除独立设备参与导致的模型偏差;其次,推导严格的SFL收敛界,评估异构设备对性能提升的贡献,指导租户激励策略;再者,将租户间设备竞争建模为拥堵博弈并分析斯塔克尔伯格均衡,推导各租户最优激励策略。在四种代表性租户(ViT、BERT、Whisper、LLaMA)及文本、图像、音频多模态数据上的大量仿真表明,相较于现有最优方法,PRINCE可将大模型微调速度提升最高达3.07倍,且始终满足微调性能目标。

原文摘要 · Abstract (English)

Foundation models (FMs) such as GPT-4 exhibit exceptional generative capabilities across diverse downstream tasks through fine-tuning. Split Federated Learning (SFL) facilitates privacy-preserving FM fine-tuning on resource-constrained local devices by offloading partial FM computations to edge servers, enabling device-edge synergistic fine-tuning. Practical edge networks often host multiple SFL tenants to support diversified downstream tasks. However, existing research primarily focuses on single-tenant SFL scenarios, and lacks tailored incentive mechanisms for multi-tenant settings, which are essential to effectively coordinate self-interested local devices for participation in various downstream tasks, ensuring that each SFL tenant's distinct FM fine-tuning requirements (e.g., FM types, performance targets, and fine-tuning deadlines) are met. To address this gap, we propose a novel Price-Incentive Mechanism (PRINCE) that guides multiple SFL tenants to offer strategic price incentives, which solicit high-quality device participation for efficient FM fine-tuning. Specifically, we first develop a bias-resilient global SFL model aggregation scheme to eliminate model biases caused by independent device participation. We then derive a rigorous SFL convergence bound to evaluate the contributions of heterogeneous devices to FM performance improvements, guiding the incentive strategies of SFL tenants. Furthermore, we model inter-tenant device competition as a congestion game for Stackelberg equilibrium (SE) analysis, deriving each SFL tenant's optimal incentive strategy. Extensive simulations involving four representative SFL tenant types (ViT, BERT, Whisper, and LLaMA) across diverse data modalities (text, images, and audio) demonstrate that PRINCE accelerates FM fine-tuning by up to 3.07x compared to state-of-the-art approaches, while consistently meeting fine-tuning performance targets.

联邦学习边缘计算大模型微调激励机制

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。