arXiv:2512.22675cs.LG2025-12

提出高效去中心化多任务学习方法,通信成本与精度无关。

Beyond Centralization: Provable Communication Efficient Decentralized Multi-Task Learning

  • 用交替投影梯度法优化低秩特征矩阵
  • 通信复杂度不随精度提升而增加,显著降耗
  • 适合数据分散、通信受限的分布式场景

表示学习是数据稀缺环境下广泛采用的框架,旨在从相关任务中提取共性特征。尽管集中式方法研究充分,去中心化方法仍较少被探索。本文研究具有低秩结构的去中心化多任务表示学习,多个任务各自拥有有限样本,观测服从线性模型且参数任务特异。在去中心化设置下,任务数据分布在多个节点上,节点间信息交换受通信网络约束。目标是恢复一个秩远小于参数维度和任务数量的底层特征矩阵。我们提出一种新的交替投影梯度与最小化算法,并提供可证明的精度保证。全面刻画了时间、通信和样本复杂度。关键优势在于通信复杂度与目标精度无关,相比以往方法大幅降低通信开销。数值模拟验证了理论分析,不同维度与网络拓扑下均有效,且在某些场景中去中心化学习优于集中式联邦方法。

原文摘要 · Abstract (English)

Representation learning is a widely adopted framework for learning in data-scarce environments, aiming to extract common features from related tasks. While centralized approaches have been extensively studied, decentralized methods remain largely underexplored. We study decentralized multi-task representation learning in which the features share a low-rank structure. We consider multiple tasks, each with a finite number of data samples, where the observations follow a linear model with task-specific parameters. In the decentralized setting, task data are distributed across multiple nodes, and information exchange between nodes is constrained by a communication network. The goal is to recover the underlying feature matrix whose rank is much smaller than both the parameter dimension and the number of tasks. We propose a new alternating projected gradient and minimization algorithm with provable accuracy guarantees. We provide comprehensive characterizations of the time, communication, and sample complexities. Importantly, the communication complexity is independent of the target accuracy, which significantly reduces communication cost compared to prior methods. Numerical simulations validate the theoretical analysis across different dimensions and network topologies, and demonstrate regimes in which decentralized learning outperforms centralized federated approaches.

去中心化多任务学习低秩通信效率

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。