去中心化联邦学习对坏数据有天然鲁棒性,尤其在坏数据集中时更稳定。
The Built-In Robustness of Decentralized Federated Averaging to Bad Data
- 通过节点间模型平均实现去中心化训练,无需中央控制
- 即使关键节点数据被污染,整体模型性能仍保持稳定
- 适合隐私敏感、网络复杂且数据质量不均的场景
去中心化联邦学习(DFL)使设备能在无中心控制器的情况下,通过复杂网络拓扑协作训练模型,本地数据保持私密,但其质量和数量在各节点间差异显著。现有研究未明确完全去中心化系统对低质或污染数据的脆弱性。由于缺乏统一错误检测与修正机制,且各节点仅掌握局部数据分布,难以判断自身视角是否反映真实分布;劣质数据训练出的模型可能在网络中传播并放大误差。为探究低质数据的影响,我们使用去中心化版本的FedAvg,在两种情形下进行模拟:一是污染数据均匀分布在部分节点,二是集中在单个节点。结果显示,基于平均的去中心化学习对局部坏数据具有显著鲁棒性,即便污染数据位于网络中最关键的节点也如此。反直觉的是,当污染数据集中于单一节点时,鲁棒性反而更强,无论该节点在网络中的中心性如何。这一现象归因于平均过程:无论节点多重要,都无法过度影响整体学习进程。
原文摘要 · Abstract (English)
Decentralized federated learning (DFL) enables devices to collaboratively train models over complex network topologies without relying on a central controller. In this setting, local data remains private, but its quality and quantity can vary significantly across nodes. The extent to which a fully decentralized system is vulnerable to poor-quality or corrupted data remains unclear, but several factors could contribute to potential risks. Without a central authority, there can be no unified mechanism to detect or correct errors, and each node operates with a localized view of the data distribution, making it difficult for the node to assess whether its perspective aligns with the true distribution. Moreover, models trained on low-quality data can propagate through the network, amplifying errors. To explore the impact of low-quality data on DFL, we simulate two scenarios with degraded data quality -- one where the corrupted data is evenly distributed in a subset of nodes and one where it is concentrated on a single node -- using a decentralized implementation of FedAvg. Our results reveal that averaging-based decentralized learning is remarkably robust to localized bad data, even when the corrupted data resides in the most influential nodes of the network. Counterintuitively, this robustness is further enhanced when the corrupted data is concentrated on a single node, regardless of its centrality in the communication network topology. This phenomenon is explained by the averaging process, which ensures that no single node -- however central -- can disproportionately influence the overall learning process.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。