动态分组源任务，提升多任务提示调优效果
Dynamic Task Vector Grouping for Efficient Multi-Task Prompt Tuning
- 用任务向量衡量相似性，动态分组最优源任务组合
- 在26个NLP数据集上实现当前最佳性能，减少负迁移
- 适合需要高效多任务学习的NLP研究者
多任务提示调优利用多个高资源源任务提升低资源目标任务的表现。现有方法通常一次性传递由所有源任务或单一‘高相似’源任务训练得到的软提示。然而我们发现，最优迁移性能往往来自源任务的特定组合，既非全部也非单一。此外，在迁移后微调过程中，源任务与目标任务的相似性会动态变化，初始阶段的相似性计算不足。为此，我们提出动态任务向量分组（DTVG）方法，核心思想包括：(1) 使用任务向量而非软提示衡量任务相似性；(2) 基于目标相似性和知识一致性两个指标，分组最优源任务组合；(3) 在每次迭代中动态更新组合。在26个NLP数据集、不同设置下的大量实验表明，DTVG能有效分组相似源任务，减少负迁移，达到当前最佳性能。
原文摘要 · Abstract (English)
Multi-task prompt tuning utilizes multiple high-resource source tasks to improve performance on low-source target tasks. Existing approaches transfer the soft prompt trained by combining all source tasks or a single ``high-similar'' source task one-time-only. However, we find that the optimal transfer performance often comes from a combination of source tasks, which is neither one nor all. Further, we find that the similarity between source and target tasks also changes dynamically during fine-tuning after transfering, making similarity calculation in the initiation stage inadequate. To address these issues, we propose a method called Dynamic Task Vector Grouping (DTVG), whose core ideas contain (1) measuring the task similarity with task vectors instead of soft prompt, (2) grouping the optimal source task combination based on two metrics: {\it target similarity} and {\it knowledge consistency}; (3) dynamically updating the combination in each iteration step. Extensive experiments on the 26 NLP datasets under different settings demonstrate that DTVG effectively groups similar source tasks while reducing negative transfer, achieving the start-of-art performance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。