用非光滑正则化让多任务图学习更高效,适合有结构关联的任务。
A non-smooth regularization framework for learning over multitask graphs
- 引入非光滑正则项,促进任务间分段常数的关联模式。
- 在凸条件下,算法均方误差逼近最优解,收敛速度为O(μ)。
- 支持ℓ₀、ℓ₁和弹性网等正则形式,计算高效且适用广泛。
本文研究多任务图上的学习问题,其中每个代理需估计自身参数向量。尽管各任务目标不同,但存在任务关联时协作可提升性能。现有方法多采用平滑正则化以实现图平滑,本文提出非光滑正则化,通过引入稀疏性,更有效地促进相邻任务间的分段常数过渡。我们构建全局正则化优化问题,最小化个体代价总和并加入通用非光滑项以鼓励邻近任务间的分段常数关系。基于前向-后向分裂策略,设计了一种去中心化学习方法,可高效求解。在成本函数与共正则化均为凸的假设下,证明该方法在均方误差意义下收敛至全局最优解的O(μ)范围内。为进一步提升适用性与效率,推导了常见非光滑(可能非凸)正则项如加权ℓ₀-范数、ℓ₁-范数及弹性网的闭式表达式。仿真验证了理论结果的有效性。
原文摘要 · Abstract (English)
In this work, we consider learning over multitask graphs, where each agent aims to estimate its own parameter vector. Although agents seek distinct objectives, collaboration among them can be beneficial in scenarios where relationships between tasks exist. Among the various approaches to promoting relationships between tasks and, consequently, enhancing collaboration between agents, one notable method is regularization. While previous multitask learning studies have focused on smooth regularization to enforce graph smoothness, this work explores non-smooth regularization techniques that promote sparsity, making them particularly effective in encouraging piecewise constant transitions on the graph. We begin by formulating a global regularized optimization problem, which involves minimizing the aggregate sum of individual costs, regularized by a general non-smooth term designed to promote piecewise-constant relationships between the tasks of neighboring agents. Based on the forward-backward splitting strategy, we propose a decentralized learning approach that enables efficient solutions to the regularized optimization problem. Then, under convexity assumptions on the cost functions and co-regularization, we establish that the proposed approach converges in the mean-square-error sense within $O(μ)$ of the optimal solution of the globally regularized cost. For broader applicability and improved computational efficiency, we also derive closed-form expressions for commonly used non-smooth (and, possibly, non-convex) regularizers, such as the weighted sum of the $\ell_0$-norm, $\ell_1$-norm, and elastic net regularization. Finally, we illustrate both the theoretical findings and the effectiveness of the approach through simulations.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。