让手机端语言模型训练更省电省带宽，还能保持效果。
CAFL-L: Constraint-Aware Federated Learning with Lagrangian Dual Optimization for On-Device Language Models
- 用拉格朗日对偶法动态调节训练参数，自动适应设备资源限制。
- 内存占用降20%，通信量减少95%，验证效果与普通方法相当。
- 适合在手机等边缘设备上部署语言模型的场景。
我们提出一种基于拉格朗日对偶优化的约束感知联邦学习方法（CAFL-L），是对经典FedAvg的改进。该方法显式考虑设备级资源约束，包括能耗、通信、内存和热预算。通过拉格朗日对偶优化，动态调整冻结层数、本地迭代步数、批大小和通信压缩策略，并利用梯度累积保障令牌预算稳定，维持训练稳定性。在字符级语言模型上的实验表明，相比标准FedAvg，CAFL-L显著提升约束满足能力：内存使用降低20%，通信量减少95%，同时保持竞争力的验证性能，适用于资源受限边缘设备的实际部署。
原文摘要 · Abstract (English)
We introduce Constraint-Aware Federated Learning with Lagrangian Dual Optimization (CAFL-L), a principled extension of FedAvg that explicitly incorporates device-level resource constraints including energy, communication, memory, and thermal budgets. CAFL-L employs Lagrangian dual optimization to dynamically adapt training hyperparameters -- freezing depth, local steps, batch size, and communication compression -- while preserving training stability through token-budget preservation via gradient accumulation. Experiments on a character-level language model demonstrate that CAFL-L achieves superior constraint satisfaction compared to standard FedAvg (reducing memory usage by 20% and communication by 95%) while maintaining competitive validation performance, making it practical for deployment on resource-constrained edge devices.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。