预测多任务学习中任务组合的收益,提升分组效率。
Ensemble Prediction of Task Affinity for Efficient Multi-Task Learning
- 用参数更新相似性计算任务对的亲和度,线性扩展至多任务组。
- 通过少量真实训练数据训练非线性模型,修正预测误差。
- 适用于大规模任务分组,提升多任务学习整体性能。
多任务学习中的核心问题是识别应联合训练的任务组。由于对所有可能的任务组合进行训练在任务数量庞大时开销过高,高效有效的任务分组关键在于预测某组任务联合学习是否能带来性能提升(相对于单任务学习的每任务增益)。本文提出ETAP(集成任务亲和力预测器),一种可扩展的框架,融合原理性与数据驱动的估计方法以预测多任务学习增益。首先,基于共享参数的梯度更新,通过任务间参数更新的相似性衡量任务对的亲和度,该线性估计器可自然扩展至多任务组。其次,为优化估计,我们训练非线性预测器,通过非线性变换与残差校正捕捉复杂任务关系;这些预测器在有限数量的真实任务组上训练,其真实增益值通过各组的多任务学习获得。实验表明,ETAP在基准数据集上显著提升多任务增益预测能力,并实现更优的任务分组,优于多个跨领域应用的现有基线方法。
原文摘要 · Abstract (English)
A fundamental problem in multi-task learning (MTL) is identifying groups of tasks that should be learned together. Since training MTL models for all possible combinations of tasks is prohibitively expensive for large task sets, a crucial component of efficient and effective task grouping is predicting whether a group of tasks would benefit from learning together, measured as per-task performance gain over single-task learning. In this paper, we propose ETAP (Ensemble Task Affinity Predictor), a scalable framework that integrates principled and data-driven estimators to predict MTL performance gains. First, we consider the gradient-based updates of shared parameters in an MTL model to measure the affinity between a pair of tasks as the similarity between the parameter updates based on these tasks. This linear estimator, which we call affinity score, naturally extends to estimating affinity within a group of tasks. Second, to refine these estimates, we train predictors that apply non-linear transformations and correct residual errors, capturing complex and non-linear task relationships. We train these predictors on a limited number of task groups for which we obtain ground-truth gain values via multi-task learning for each group. We demonstrate on benchmark datasets that ETAP improves MTL gain prediction and enables more effective task grouping, outperforming state-of-the-art baselines across diverse application domains.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。