提出抗拜占庭攻击的零阶优化方法，大幅降低通信开销。
Byzantine-Resilient Zero-Order Optimization for Communication-Efficient Heterogeneous Federated Learning
- 采用变换鲁棒聚合，适应数据异构下的非凸优化
- 每轮仅需少量标量通信，内存占用显著减少
- 适用于资源受限场景，尤其适合大模型微调
我们提出CyBeR-0，一种抗拜占庭攻击的联邦零阶优化方法，在客户端数据异构条件下仍能保证收敛，并显著降低上行与下行通信成本。通过引入变换鲁棒聚合，该方法为一般非凸目标提供了收敛性保障。在标准学习任务和大语言模型微调的实验中，CyBeR-0仅需每轮少量标量通信，即可保持稳定性能，同时大幅降低内存需求。
原文摘要 · Abstract (English)
We introduce CyBeR-0, a Byzantine-resilient federated zero-order optimization method that is robust under Byzantine attacks and provides significant savings in uplink and downlink communication costs. We introduce transformed robust aggregation to give convergence guarantees for general non-convex objectives under client data heterogeneity. Empirical evaluations for standard learning tasks and fine-tuning large language models show that CyBeR-0 exhibits stable performance with only a few scalars per-round communication cost and reduced memory requirements.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。