用几何方法统一异构联邦学习中的表示,无需共享空间却能保持一致性。
Sheaf-Based Federated Representation Learning

- 基于可学习的层化映射,通过正交变换对齐邻近节点表示
- 在少量共享样本上计算惩罚项,通信开销小且精度高
- 适合数据/模型异构场景,尤其抗压缩能力强
异构联邦系统中,各参与方的数据分布、感知模态、模型结构、隐变量维度和局部目标各不相同,难以学习并交换有效表示。为此,我们提出基于层(Sheaf)的联邦表示学习(SFRL)框架,联合优化本地目标与基于可学习层限制映射的流形约束几何对齐正则项。不同于多数现有方法假设存在共享全局隐空间,SFRL中全局一致性由相邻隐表示通过正交变换和保距嵌入对齐自然涌现。该对齐由层拉普拉斯诱导的二次粘合正则项强制实现,其可学习的限制映射能自适应数据几何。惩罚项仅在少量共享引导样本上评估,确保可扩展性与通信效率。我们设计了一种去中心化算法 Sheaf-FRL,交替进行本地模型梯度更新与边级限制映射的闭式 Procrustes 更新。进一步证明了 Sheaf-FRL 在确定性和随机设置下均收敛至一阶驻点。作为应用,我们在语义通信背景下考虑协作分类任务,面对模型与数据异构性。结果表明,无论局部分布偏移程度如何,Sheaf-FRL 在本地及通信后分类准确率上均优于基线方法,且对隐空间维数压缩具有更强鲁棒性。
原文摘要 · Abstract (English)
Heterogeneous federated systems require agents to learn and exchange informative representations despite differences in data distributions, sensing modalities, model architectures, latent dimensionalities, and local learning objectives. To address this challenge, we propose Sheaf-based Federated Representation Learning (SFRL), a general framework that jointly optimizes local objectives with a manifold-constrained geometric alignment regularizer based on learnable sheaf restriction maps. Unlike most existing approaches, SFRL does not assume a shared global latent space. Instead, global consistency emerges from the alignment of neighboring latent representations through orthogonal transformations and isometric embeddings. This alignment is enforced by a quadratic gluing regularizer induced by the sheaf Laplacian, whose learnable restriction maps adapt the geometry to the observed data. The penalty is evaluated on a small set of shared pilot samples, ensuring scalability and communication efficiency. We develop a decentralized algorithm for solving SFRL, termed Sheaf-FRL, which alternates between gradient updates of the local models and closed-form Procrustes updates of the edge-wise restriction maps. We further establish convergence of Sheaf-FRL to first-order stationary points in both deterministic and stochastic settings. As an application, we consider a cooperative classification task in the context of semantic communication, under model and data heterogeneity. Our results show that Sheaf-FRL outperforms baseline approaches in terms of local and post-communication classification accuracy across different levels of local distribution shift and exhibits greater robustness to latent-space dimensionality compression.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。