arXiv:2504.09812cs.LGcs.AI2025-04

无需手动设计,自动融合单任务模型构建多任务系统

Efficient Multi-Task Modeling through Automated Fusion of Trained Models

  • 将不同结构的单任务模型分解为可操作组件
  • 用Transformer架构自适应融合任务内与任务间知识
  • 简化流程,适合快速搭建多任务应用

尽管多任务学习广泛应用于智能服务,传统方法通常需针对特定任务组合定制设计,建模过程繁琐。本文提出一种高效多任务建模方法,可自动融合已训练好的异构单任务模型,形成多任务模型。该框架只需准备各任务的已训练模型,即可简化建模流程,充分保留原有知识,避免过度关注任务关系与模型结构设计。为此,我们考虑不同模型间的结构差异,采用模型分解技术将其分层拆解为多个可操作组件,并设计基于Transformer的自适应知识融合(AKF)模块,根据组件动态整合任务内与任务间知识。通过在三个数据集上的大量实验验证,该方法实现了多任务模型的高效自动化构建。

原文摘要 · Abstract (English)

Although multi-task learning is widely applied in intelligent services, traditional multi-task modeling methods often require customized designs based on specific task combinations, resulting in a cumbersome modeling process. Inspired by the rapid development and excellent performance of single-task models, this paper proposes an efficient multi-task modeling method that can automatically fuse trained single-task models with different structures and tasks to form a multi-task model. As a general framework, this method allows modelers to simply prepare trained models for the required tasks, simplifying the modeling process while fully utilizing the knowledge contained in the trained models. This eliminates the need for excessive focus on task relationships and model structure design. To achieve this goal, we consider the structural differences among various trained models and employ model decomposition techniques to hierarchically decompose them into multiple operable model components. Furthermore, we have designed an Adaptive Knowledge Fusion (AKF) module based on Transformer, which adaptively integrates intra-task and inter-task knowledge based on model components. Through the proposed method, we achieve efficient and automated construction of multi-task models, and its effectiveness is verified through extensive experiments on three datasets.

多任务学习模型融合自动化建模

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。