arXiv:2603.12976cs.LGcs.CV2026-03

联邦学习中用语义核心集减少数据偏移,提升长尾分布下的模型性能。

SCOPE: Semantic Coreset with Orthogonal Projection Embeddings for Federated learning

  • 基于正交投影嵌入构建全局语义核心集,动态过滤噪声与冗余数据。
  • 在极端类别不平衡下实现与全数据训练相当的准确率,上行带宽降低128至512倍。
  • 仅传输标量指标,适合通信资源受限的分布式科研场景。

科学发现日益依赖来自高分辨率仪器流数据的联邦学习,这些数据存在极端类别不平衡问题。现有机器学习方法要么需要不切实际的数据聚合,要么因类别不平衡而失效。现有核心集选择方法依赖局部启发式策略,无法感知全局数据分布,易导致次优且非代表性的删减。为此,我们提出SCOPE(Semantic Coreset using Orthogonal Projection Embeddings for Federated learning),一种面向联邦数据的核心集框架,通过过滤异常值并自适应剔除冗余数据来缓解长尾偏斜。通过分析隐空间分布,我们为每个数据点计算三类得分:表示得分(衡量核心类别特征的可靠性)、多样性得分(量化正交残差的新颖性)和边界接近得分(指示与竞争类别的相似度)。与以往方法不同,SCOPE仅向联邦服务器传输标量指标以构建全局共识,确保通信高效。在全局共识指导下,SCOPE动态过滤本地噪声并丢弃冗余样本,以应对全局特征偏斜。大量实验表明,SCOPE在保持优异全局准确率和鲁棒收敛的同时,实现上行带宽降低128至512倍、壁钟时间加速7.72倍,并显著减少本地核心集选择的浮点运算和显存占用。

原文摘要 · Abstract (English)

Scientific discovery increasingly requires learning on federated datasets, fed by streams from high-resolution instruments, that have extreme class imbalance. Current ML approaches either require impractical data aggregation or fail due to class imbalance. Existing coreset selection methods rely on local heuristics, making them unaware of the global data landscape and prone to sub-optimal and non-representative pruning. To overcome these challenges, we introduce SCOPE (Semantic Coreset using Orthogonal Projection Embeddings for Federated learning), a coreset framework for federated data that filters anomalies and adaptively prunes redundant data to mitigate long-tail skew. By analyzing the latent space distribution, we score each data point using a representation score that measures the reliability of core class features, a diversity score that quantifies the novelty of orthogonal residuals, and a boundary proximity score that indicates similarity to competing classes. Unlike prior methods, SCOPE shares only scalar metrics with a federated server to construct a global consensus, ensuring communication efficiency. Guided by the global consensus, SCOPE dynamically filters local noise and discards redundant samples to counteract global feature skews. Extensive experiments demonstrate that SCOPE yields competitive global accuracy and robust convergence, all while achieving exceptional efficiency with a 128x to 512x reduction in uplink bandwidth, a 7.72x wall-clock acceleration and reduced FLOP and VRAM footprints for local coreset selection.

联邦学习核心集长尾分布通信效率

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。