用能量距离量化数据异构性，提升分布式学习收敛效率
Measuring Heterogeneity in Machine Learning with Distributed Energy Distance
- 引入能量距离衡量特征分布差异，敏感且鲁棒
- 通过泰勒近似降低计算开销，适用于大规模系统
- 可动态调整节点惩罚权重，改善联邦学习协作
在分布式和联邦学习中，数据源间的异构性仍是有效模型聚合与收敛的主要障碍。本文聚焦特征异构性，引入能量距离作为量化分布差异的敏感指标。尽管能量距离对数据分布偏移具有鲁棒检测能力，但其直接应用在大规模系统中计算成本过高。为此，我们提出泰勒近似方法，在保留关键理论性质的同时显著降低计算开销。模拟实验表明，准确捕捉特征差异能有效提升分布式学习的收敛性能。最后，我们提出一种新应用：利用能量距离为异构节点的预测结果分配惩罚权重，从而增强联邦与分布式环境下的协同效果。
原文摘要 · Abstract (English)
In distributed and federated learning, heterogeneity across data sources remains a major obstacle to effective model aggregation and convergence. We focus on feature heterogeneity and introduce energy distance as a sensitive measure for quantifying distributional discrepancies. While we show that energy distance is robust for detecting data distribution shifts, its direct use in large-scale systems can be prohibitively expensive. To address this, we develop Taylor approximations that preserve key theoretical quantitative properties while reducing computational overhead. Through simulation studies, we show how accurately capturing feature discrepancies boosts convergence in distributed learning. Finally, we propose a novel application of energy distance to assign penalty weights for aligning predictions across heterogeneous nodes, ultimately enhancing coordination in federated and distributed settings.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。