arXiv:2409.11509cs.LGcs.AI2024-09NeurIPS被引 2

联邦学习下实现高效邻居嵌入降维,无需共享原始数据

FedNE: Surrogate-Assisted Federated Neighbor Embedding for Dimensionality Reduction

  • 用代理损失函数解决联邦环境下的局部排斥不足问题
  • 在合成与真实数据集上均保持邻域结构并提升全局对齐效果
  • 适合需要可视化联邦数据的科研与工业场景

联邦学习(FL)作为一种新兴范式,可在不交换本地数据的前提下实现多方协同建模。尽管已在计算机视觉、图学习和自然语言处理等领域广泛应用,但如何在联邦环境下有效构建数据投影模型以实现数据可视化仍严重缺乏研究。邻居嵌入(NE)是高维数据可视化的关键技术,但协作学习联合NE模型面临挑战。核心难点在于目标函数:如NE等有效可视化算法需计算数据对间的损失。本文提出 extsc{FedNE},将 extsc{FedAvg}框架与对比式邻居嵌入结合,无需共享可辨识数据。为解决全局嵌入空间对齐所需的客户端间排斥缺失问题,设计了客户端各自学习并共享的代理损失函数。此外,提出数据混合策略,通过增强本地数据缓解局部kNN图导致的不可见邻居与虚假邻居问题。在合成与真实数据集上进行充分实验,结果表明 extsc{FedNE}能有效保留邻域结构,并显著提升全局嵌入空间对齐性,优于多个基线方法。

原文摘要 · Abstract (English)

Federated learning (FL) has rapidly evolved as a promising paradigm that enables collaborative model training across distributed participants without exchanging their local data. Despite its broad applications in fields such as computer vision, graph learning, and natural language processing, the development of a data projection model that can be effectively used to visualize data in the context of FL is crucial yet remains heavily under-explored. Neighbor embedding (NE) is an essential technique for visualizing complex high-dimensional data, but collaboratively learning a joint NE model is difficult. The key challenge lies in the objective function, as effective visualization algorithms like NE require computing loss functions among pairs of data. In this paper, we introduce \textsc{FedNE}, a novel approach that integrates the \textsc{FedAvg} framework with the contrastive NE technique, without any requirements of shareable data. To address the lack of inter-client repulsion which is crucial for the alignment in the global embedding space, we develop a surrogate loss function that each client learns and shares with each other. Additionally, we propose a data-mixing strategy to augment the local data, aiming to relax the problems of invisible neighbors and false neighbors constructed by the local $k$NN graphs. We conduct comprehensive experiments on both synthetic and real-world datasets. The results demonstrate that our \textsc{FedNE} can effectively preserve the neighborhood data structures and enhance the alignment in the global embedding space compared to several baseline methods.

联邦学习降维可视化邻居嵌入

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。