通过极值聚类提升联邦学习的统计精度与压缩效率
Range Penalization: Theoretical Insights with Applications in Federated Learning
- 识别跨客户端共享特征，对个性化特征进行极值聚类
- 理论证明可实现精准模式恢复，且不依赖渐近分析
- 适合关注模型压缩、资源受限场景的联邦学习研究者
本文针对具有线性系统成分的联邦学习提出范围正则化方法，以提升统计精度并促进跨客户端规则性，有利于量化、编码与资源高效。该方法识别不同客户端间的共享特征，并自适应地将个性化特征权重聚类至极端值，称为极值聚类。由于正则项具有半范数性质且不可分解，相关估计器的理论分析面临挑战。我们发展了新的非渐近分析证明技术，确保统计精度与模式恢复的可靠性。此外，提出一种利用局部强凸性差异的快速优化算法，降低迭代复杂度。实验验证了方法的有效性与高效性。
原文摘要 · Abstract (English)
This paper introduces range regularization for federated learning with linear systematic components to enhance statistical accuracy and induce cross-client regularity conducive to quantization, coding, and resource efficiency. Our approach identifies features with shared weights across different clients and adaptively clusters the weights of personalized features at extreme values, a process we refer to as polar clustering. Theoretical analysis of the associated estimators poses significant challenges due to the seminorm nature and non-decomposability of the regularizer. We develop new proof techniques for the nonasymptotic analysis of statistical accuracy and faithful pattern recovery. Moreover, a fast optimization algorithm that leverages varying degrees of local strong convexity is proposed to reduce iteration complexity. Experiments support the efficacy and efficiency of the proposed approach.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。