用数据归属度量化任务细粒度相关性，缓解多任务学习负迁移。
Measuring Fine-Grained Relatedness in Multitask Learning via Data Attribution
- 基于影响函数扩展出多任务影响力函数（MTIF），支持硬/软参数共享。
- 实例级相关性度量显著提升负迁移缓解效果，性能优于传统方法。
- 适合关注多任务学习优化与数据选择的研究者使用。
衡量任务相关性并缓解负迁移仍是多任务学习（MTL）中的关键挑战。本文将数据归属度——用于量化单个训练样本对模型预测影响的方法——拓展至MTL场景，提出多任务影响函数（MTIF），可适配具有硬共享或软共享参数的MTL模型。相较于传统整体任务层面的相关性度量，MTIF提供细粒度的实例级相关性评估。该度量支持一种数据选择策略，有效缓解多任务学习中的负迁移。大量实验表明，所提MTIF能高效准确地近似基于子集数据训练模型的性能；基于MTIF的数据选择策略在各类MTL任务中均持续提升模型表现。本工作建立了数据归属度与MTL之间的新关联，为任务相关性测量与多任务学习性能提升提供了高效、细粒度的解决方案。
原文摘要 · Abstract (English)
Measuring task relatedness and mitigating negative transfer remain a critical open challenge in Multitask Learning (MTL). This work extends data attribution -- which quantifies the influence of individual training data points on model predictions -- to MTL setting for measuring task relatedness. We propose the MultiTask Influence Function (MTIF), a method that adapts influence functions to MTL models with hard or soft parameter sharing. Compared to conventional task relatedness measurements, MTIF provides a fine-grained, instance-level relatedness measure beyond the entire-task level. This fine-grained relatedness measure enables a data selection strategy to effectively mitigate negative transfer in MTL. Through extensive experiments, we demonstrate that the proposed MTIF efficiently and accurately approximates the performance of models trained on data subsets. Moreover, the data selection strategy enabled by MTIF consistently improves model performance in MTL. Our work establishes a novel connection between data attribution and MTL, offering an efficient and fine-grained solution for measuring task relatedness and enhancing MTL models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。