用几何建模解决联邦学习个性化难题,提升异构数据下性能与隐私。
Variational Gaussian Mixture Manifold Models for Client-Specific Federated Personalization
- 基于变分高斯混合流形,为每客户端学习专属嵌入空间。
- 仅交换不确定性感知的统计摘要,通信量小且在非独立同分布数据上表现优。
- 适合需要个性化、低通信开销和强隐私保护的联邦学习场景。
个性化联邦学习(PFL)在标签偏斜和非平稳环境下常因单一全局参数化而失效。本文提出VGM²(变分高斯混合流形),一种以几何为中心的PFL框架:(i) 学习客户端特定的参数化UMAP嵌入;(ii) 用混合关系标记建模同类与异类样本间的潜在距离;(iii) 仅交换变分化的、不确定性感知的标记统计量。每个客户端维护一个狄利克雷-正态-逆伽马(Dir-NIG)后验分布,包含标记权重、均值和方差;服务器通过共轭矩匹配聚合,形成全局先验以指导后续轮次。我们证明该聚合在共轭族内最小化客户后验的反向KL散度之和,确保异构环境下的稳定性。进一步引入距离到相似度映射的校准项,并报告通信与计算预算。在八个视觉数据集(含非独立同分布标签分片)上,VGM²在测试F1得分上优于或媲美强基线,同时仅传输少量几何摘要。通过安全聚合与可选差分隐私噪声强化隐私,提供成员推理压力测试。代码与配置将公开以保证可复现性。
原文摘要 · Abstract (English)
Personalized federated learning (PFL) often fails under label skew and non-stationarity because a single global parameterization ignores client-specific geometry. We introduce VGM$^2$ (Variational Gaussian Mixture Manifold), a geometry-centric PFL framework that (i) learns client-specific parametric UMAP embeddings, (ii) models latent pairwise distances with mixture relation markers for same and different class pairs, and (iii) exchanges only variational, uncertainty-aware marker statistics. Each client maintains a Dirichlet-Normal-Inverse-Gamma (Dir-NIG) posterior over marker weights, means, and variances; the server aggregates via conjugate moment matching to form global priors that guide subsequent rounds. We prove that this aggregation minimizes the summed reverse Kullback-Leibler divergence from client posteriors within the conjugate family, yielding stability under heterogeneity. We further incorporate a calibration term for distance-to-similarity mapping and report communication and compute budgets. Across eight vision datasets with non-IID label shards, VGM$^2$ achieves competitive or superior test F1 scores compared to strong baselines while communicating only small geometry summaries. Privacy is strengthened through secure aggregation and optional differential privacy noise, and we provide a membership-inference stress test. Code and configurations will be released to ensure full reproducibility.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。