提出高效一阶算法，实现多任务共享线性表征的近优学习
Near-optimal and Efficient First-Order Algorithm for Multi-Task Learning with Shared Linear Representation
- 基于一阶优化联合学习共享表征与任务参数
- 收敛速度$ ilde{ ext{O}}(1)$次迭代，误差$ ilde{ ext{O}}(dk/(TN))$
- 相比已有方法提升$k$倍，适合大规模多任务场景
多任务学习（MTL）通过利用多个相关任务间的共享结构成为机器学习的关键范式。尽管其在实践中取得成功，但针对共享线性表示的似然基础上可高效求解的算法仍不充分，主要源于矩阵分解固有的非凸性。本文提出一种一阶算法，联合学习共享表示与任务特定参数，并保证计算效率。该算法在$ ilde{ ext{O}}(1)$次迭代内收敛，达到$ ilde{ ext{O}}(dk/(TN))$的近优估计误差，较现有似然方法提升$k$倍，其中$d$为输入维度，$k$为表示维度，$T$为任务数，$N$为每任务样本数。结果表明，似然基础的一阶方法可高效解决MTL问题。
原文摘要 · Abstract (English)
Multi-task learning (MTL) has emerged as a pivotal paradigm in machine learning by leveraging shared structures across multiple related tasks. Despite its empirical success, the development of likelihood-based efficiently solvable algorithms--even for shared linear representations--remains largely underdeveloped, primarily due to the non-convex structure intrinsic to matrix factorization. This paper introduces a first-order algorithm that jointly learns a shared representation and task-specific parameters, with guaranteed efficiency. Notably, it converges in $\widetilde{\mathcal{O}}(1)$ iterations and attains a \emph{near-optimal} estimation error of $\widetilde{\mathcal{O}}(dk/(TN))$, \emph{improving} over existing likelihood-based methods by a factor of $k$, where $d$, $k$, $T$, $N$ denote input dimension, representation dimension, task count, and samples per task, respectively. Our results justify that likelihood-based first-order methods can efficiently solve the MTL problem.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。