提出一种去中心化多任务表示学习算法，高效共享低维特征。
Diffusion-based Decentralized Federated Multi-Task Representation Learning
- 基于扩散式去中心化框架，用投影梯度下降交替优化特征矩阵。
- 理论证明算法在有限样本下可收敛，且迭代次数与通信开销均较低。
- 适合数据分散、需共享特征的分布式机器学习场景。
表示学习是数据稀缺环境下获取通用特征提取器的重要方法，尤其适用于多个相关任务。尽管该领域研究广泛，去中心化方法仍相对不足。本文针对多任务线性回归问题，提出一种基于扩散式去中心化联邦架构的投影梯度下降算法。多个线性回归模型共享一个低秩线性表示。通过交替最小化与投影梯度下降，实现低秩特征矩阵的恢复。我们提供了可证明的保证：给出了所需样本复杂度的下界和迭代复杂度的上界。分析了算法的时间与通信复杂度，表明其高效且通信成本低。数值实验验证了算法性能，并与基准方法进行了对比。
原文摘要 · Abstract (English)
Representation learning is a widely adopted framework for learning in data-scarce environments to obtain a feature extractor or representation from various different yet related tasks. Despite extensive research on representation learning, decentralized approaches remain relatively underexplored. This work develops a decentralized projected gradient descent-based algorithm for multi-task representation learning. We focus on the problem of multi-task linear regression in which multiple linear regression models share a common, low-dimensional linear representation. We present an alternating projected gradient descent and minimization algorithm for recovering a low-rank feature matrix in a diffusion-based decentralized and federated fashion. We obtain constructive, provable guarantees that provide a lower bound on the required sample complexity and an upper bound on the iteration complexity of our proposed algorithm. We analyze the time and communication complexity of our algorithm and show that it is fast and communication-efficient. We performed numerical simulations to validate the performance of our algorithm and compared it with benchmark algorithms.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。