提出可自适应学习任务关系的多任务学习框架，提升异构数据下的模型性能。
Multitask Learning with Learned Task Relationships
- 用高斯马尔可夫随机场建模任务间关系，联合优化任务关联与本地模型。
- 理论证明学习到的任务关系质量，实验显示在异构数据下显著优于传统方法。
- 适合分布式学习中需个性化建模且无先验知识的场景，如边缘设备协同训练。
经典的联邦与去中心化学习中的共识策略在存在异构本地数据或任务分布时统计上次优。近年来，多任务或个性化策略受到关注，允许各参与方在不强制共识的前提下，基于本地最优模型相互受益。现有方法要么依赖精确的任务关系先验知识，要么完全非参数化，依赖元学习或近端构造。本文提出一种介于两者之间的算法框架：通过未知精度矩阵的高斯马尔可夫随机场建模任务关系，联合学习任务关系与局部模型，使各代理能根据自身数据分布自我组织。理论分析量化了所学关系的质量，数值实验验证了其实际有效性。
原文摘要 · Abstract (English)
Classical consensus-based strategies for federated and decentralized learning are statistically suboptimal in the presence of heterogeneous local data or task distributions. As a result, in recent years, there has been growing interest in multitask or personalized strategies, which allow individual agents to benefit from one another in pursuing locally optimal models without enforcing consensus. Existing strategies require either precise prior knowledge of the underlying task relationships or are fully non-parametric and instead rely on meta-learning or proximal constructions. In this work, we introduce an algorithmic framework that strikes a balance between these extremes. By modeling task relationships through a Gaussian Markov Random Field with an unknown precision matrix, we develop a strategy that jointly learns both the task relationships and the local models, allowing agents to self-organize in a way consistent with their individual data distributions. Our theoretical analysis quantifies the quality of the learned relationship, and our numerical experiments demonstrate its practical effectiveness.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。