解决异构数据下联邦学习的高斯混合模型估计偏差问题
Decentralized EM Algorithm for Gaussian Mixtures under Data Heterogeneity and Partial Labeling
- 引入动量网络机制融合历史与当前估计值
- 理论证明可达到全局样本最优效率，且加速收敛
- 适用于部分标注数据场景，适合医疗图像等实际应用
我们系统研究了在去中心化联邦学习（DFL）框架下基于网络的期望最大化（EM）算法在高斯混合模型中的应用。理论分析表明，当数据在各节点间分布异质时，直接扩展经典EM算法会导致估计偏差。为此，我们提出动量网络EM（MNEM）算法，融合当前及历史迭代的估计信息。进一步提出半监督MNEM（semi-MNEM）算法，利用部分标注数据提升性能。严格理论分析显示，在适当正则条件下，MNEM估计器能达到与全样本估计器相同的渐近效率，即使在数据异质情况下亦然。此外，semi-MNEM显著加快了收敛速度，即便不同混合成分分离度较差。通过大量模拟实验及广泛使用的胸部X光数据集分析，验证了所提方法的有限样本表现。
原文摘要 · Abstract (English)
We systematically study several network-based Expectation-Maximization (EM) algorithms for the Gaussian mixture model within decentralized federated learning (DFL). Our theoretical investigation shows that directly extending the classic EM algorithm to DFL leads to a biased estimator when data are heterogeneously distributed across sites. To address this, we introduce a momentum network EM (MNEM) algorithm, which integrates information from both current and historical estimators from previous DFL iterations. We further develop a semi-supervised MNEM (semi-MNEM) algorithm, which utilizes information provided by partially labeled data. Rigorous theoretical analysis demonstrates that the MNEM estimator can achieve the same asymptotic efficiency as the whole-sample estimator under appropriate regularity conditions, even with heterogeneous data. Moreover, the semi-MNEM estimator significantly improves the convergence speed of the MNEM algorithm, even if different mixture components are poorly separated. Extensive simulations are conducted, and a widely used chest X-ray dataset is analyzed to demonstrate the finite-sample performance of the proposed methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。