无需额外计算成本,用几何投影实现联邦学习个性化
Cost-Free Personalization via Information-Geometric Projection in Bayesian Federated Learning
- 通过信息几何投影将全局模型映射到局部数据邻域
- 在保持全局性能的同时显著提升本地模型效果,计算开销几乎为零
- 适合追求高效个性化的隐私保护场景,如医疗、金融
贝叶斯联邦学习(BFL)结合不确定性建模与分布式训练,在数据异构和隐私约束下实现个性化与可靠模型。现有方法多依赖马尔可夫链蒙特卡洛(MCMC)或变分推断,并引入个性化机制以适应本地数据分布。本文提出一种基于信息几何投影的参数化BFL个性化框架,通过将全局模型投影至用户局部模型的邻域,实现全局泛化与局部特化之间的可调权衡。在弱假设下,该投影等价于统计流形上的巴氏中心计算,可导出闭式解,实现无成本个性化。我们将其应用于改进的变分在线牛顿(IVON)优化器,并拓展至通用聚合方案。在异构数据下的实证评估表明,该方法有效平衡全局与本地性能,计算开销极低。
原文摘要 · Abstract (English)
Bayesian Federated Learning (BFL) combines uncertainty modeling with decentralized training, enabling the development of personalized and reliable models under data heterogeneity and privacy constraints. Existing approaches typically rely on Markov Chain Monte Carlo (MCMC) sampling or variational inference, often incorporating personalization mechanisms to better adapt to local data distributions. In this work, we propose an information-geometric projection framework for personalization in parametric BFL. By projecting the global model onto a neighborhood of the user's local model, our method enables a tunable trade-off between global generalization and local specialization. Under mild assumptions, we show that this projection step is equivalent to computing a barycenter on the statistical manifold, allowing us to derive closed-form solutions and achieve cost-free personalization. We apply the proposed approach to a variational learning setup using the Improved Variational Online Newton (IVON) optimizer and extend its application to general aggregation schemes in BFL. Empirical evaluations under heterogeneous data distributions confirm that our method effectively balances global and local performance with minimal computational overhead.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。