首次在联邦学习中定义模型多重性,揭示隐私保护下的决策差异。
Rashomon Sets and Model Multiplicity in Federated Learning
- 提出三种联邦学习下的近似最优模型集合定义
- 在隐私约束下可估计模型多样性指标
- 帮助各客户端选择更符合本地数据的模型
Rashomon集描述了在经验性能相近但决策边界差异显著的一组模型。理解这些模型间的差异——即模型多重性——是实现模型透明性、公平性和鲁棒性的关键步骤,因为它揭示了标准指标所掩盖的决策不稳定性。然而,现有Rashomon集定义和多重性度量均基于集中式学习,难以直接扩展到联邦学习(FL)这类去中心化多参与方场景。在FL中,多个客户端在中央服务器协调下联合训练模型,不共享原始数据,虽保障隐私却面临客户端数据分布异构与通信限制挑战。此时选择单一最优模型可能使不同客户端的预测行为趋同,放大偏见,破坏公平性保证。本文首次在联邦学习中形式化了Rashomon集:首先,将Rashomon集定义适配至FL,区分三类视角——(I) 全局集(基于所有客户端聚合统计),(II) t-一致集(跨分数t客户端局部集的交集),(III) 各客户端专属的局部集;其次,展示了如何在隐私约束下估计标准多重性度量;最后,设计了一个关注多样性的联邦学习流程,并在标准基准数据集上进行实证研究。结果表明,三类联邦Rashomon集定义均提供有价值洞察,使客户端能够部署更契合其本地数据、公平性考量与实际需求的模型。
原文摘要 · Abstract (English)
The Rashomon set captures the collection of models that achieve near-identical empirical performance yet may differ substantially in their decision boundaries. Understanding the differences among these models, i.e., their multiplicity, is recognized as a crucial step toward model transparency, fairness, and robustness, as it reveals decision boundaries instabilities that standard metrics obscure. However, the existing definitions of Rashomon set and multiplicity metrics assume centralized learning and do not extend naturally to decentralized, multi-party settings like Federated Learning (FL). In FL, multiple clients collaboratively train models under a central server's coordination without sharing raw data, which preserves privacy but introduces challenges from heterogeneous client data distribution and communication constraints. In this setting, the choice of a single best model may homogenize predictive behavior across diverse clients, amplify biases, or undermine fairness guarantees. In this work, we provide the first formalization of Rashomon sets in FL.First, we adapt the Rashomon set definition to FL, distinguishing among three perspectives: (I) a global Rashomon set defined over aggregated statistics across all clients, (II) a t-agreement Rashomon set representing the intersection of local Rashomon sets across a fraction t of clients, and (III) individual Rashomon sets specific to each client's local distribution.Second, we show how standard multiplicity metrics can be estimated under FL's privacy constraints. Finally, we introduce a multiplicity-aware FL pipeline and conduct an empirical study on standard FL benchmark datasets. Our results demonstrate that all three proposed federated Rashomon set definitions offer valuable insights, enabling clients to deploy models that better align with their local data, fairness considerations, and practical requirements.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。