用低秩贝叶斯修正实现个性化联邦学习，提升校准度与效率
Personalizing Low-Rank Bayesian Neural Networks Via Federated Learning
- 全局模型加低秩贝叶斯修正，降低计算存储开销
- 自适应选择秩，匹配不同客户端的不确定性水平
- 在多个数据集上兼顾精度、校准度与资源消耗
为支持现实世界决策，模型需具备良好校准性，即对预测结果给出可靠的置信度估计。在个性化联邦学习中，由于客户端本地数据量小，难以确定最优模型参数，不确定性量化尤为重要。贝叶斯联邦学习（BPFL）虽可提升校准性，但常因需追踪所有参数方差而带来显著计算与内存开销。此外，不同客户端因本地数据规模和分布差异，表现出异质性不确定性。为此，我们提出LR-BPFL，一种新型贝叶斯联邦学习方法：学习一个全局确定性模型，并结合个性化的低秩贝叶斯修正。通过自适应秩选择机制，使局部模型能适配各客户端固有的不确定性水平。我们在多种数据集上评估了该方法，结果表明其在校准性、准确率以及计算与内存开销方面均具优势。
原文摘要 · Abstract (English)
To support real-world decision-making, it is crucial for models to be well-calibrated, i.e., to assign reliable confidence estimates to their predictions. Uncertainty quantification is particularly important in personalized federated learning (PFL), as participating clients typically have small local datasets, making it difficult to unambiguously determine optimal model parameters. Bayesian PFL (BPFL) methods can potentially enhance calibration, but they often come with considerable computational and memory requirements due to the need to track the variances of all the individual model parameters. Furthermore, different clients may exhibit heterogeneous uncertainty levels owing to varying local dataset sizes and distributions. To address these challenges, we propose LR-BPFL, a novel BPFL method that learns a global deterministic model along with personalized low-rank Bayesian corrections. To tailor the local model to each client's inherent uncertainty level, LR-BPFL incorporates an adaptive rank selection mechanism. We evaluate LR-BPFL across a variety of datasets, demonstrating its advantages in terms of calibration, accuracy, as well as computational and memory requirements.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。