将模型拆成碎片分散传播，提升去中心化学习效率与准确率。
Mosaic Learning: A Framework for Decentralized Learning with Model Fragmentation
- 模型碎片化传播，降低冗余通信
- 节点测试准确率最高提升12个百分点
- 适合数据隐私要求高的分布式场景
去中心化学习（DL）允许在无中央服务器的情况下协同机器学习，适用于训练数据无法集中存储的场景。我们提出Mosaic Learning，一种将模型分解为碎片并在网络中独立传播的DL框架。碎片化减少了相关参数间的冗余通信，同时在不增加通信开销的前提下实现更丰富的信息传播。理论上证明Mosaic Learning具有最先进的最坏情况收敛速率，并利用模型参数相关性，通过降低简化系统的最大特征值来提升收缩率。我们在四个学习任务上进行了实验评估，结果显示，相较于当前最优基线方法——流行病学习（EL），Mosaic Learning在节点级测试准确率上最高提升12个百分点。总体而言，Mosaic Learning在不牺牲去中心化学习实用性与效率的前提下显著提升了性能，有望成为新的去中心化学习标准。
原文摘要 · Abstract (English)
Decentralized learning (DL) enables collaborative machine learning (ML) without a central server, making it suitable for settings where training data cannot be centrally hosted. We introduce Mosaic Learning, a DL framework that decomposes models into fragments and disseminates them independently across the network. Fragmentation reduces redundant communication across correlated parameters and enables more diverse information propagation without increasing communication cost. We theoretically show that Mosaic Learning (i) shows state-of-the-art worst-case convergence rate, and (ii) leverages parameter correlation in an ML model, improving contraction by reducing the highest eigenvalue of a simplified system. We empirically evaluate Mosaic Learning on four learning tasks and observe up to 12 percentage points higher node-level test accuracy compared to epidemic learning (EL), a state-of-the-art baseline. In summary, Mosaic Learning improves DL performance without sacrificing its utility or efficiency, and positions itself as a new DL standard.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。