研究不同相似度度量在数据分布偏移下的去中心化学习效果
On the effects of similarity metrics in decentralized deep learning under distributional shift
- 用多种相似度度量筛选可协作的模型，避免直接共享数据
- 在多个含分布偏移的数据集上验证方法有效性
- 适合关注隐私保护下模型协同优化的研究者
去中心化学习（DL）使组织或用户能在不泄露数据的前提下协作提升本地深度学习模型性能。然而当客户端数据存在异构性时，模型聚合面临挑战，且在不直接交换数据的情况下识别兼容合作者仍是难题。本文针对多种相似度度量在分布式偏移场景下的去中心化学习中用于模型合并伙伴选择的有效性进行了实证分析，覆盖多个数据集。研究揭示了各类度量的表现差异，探讨其在促进有效协作中的作用，为构建鲁棒的去中心化学习方法提供支持。
原文摘要 · Abstract (English)
Decentralized Learning (DL) enables privacy-preserving collaboration among organizations or users to enhance the performance of local deep learning models. However, model aggregation becomes challenging when client data is heterogeneous, and identifying compatible collaborators without direct data exchange remains a pressing issue. In this paper, we investigate the effectiveness of various similarity metrics in DL for identifying peers for model merging, conducting an empirical analysis across multiple datasets with distribution shifts. Our research provides insights into the performance of these metrics, examining their role in facilitating effective collaboration. By exploring the strengths and limitations of these metrics, we contribute to the development of robust DL methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。