用三阶段框架让大模型同时处理多个任务，性能接近单任务但成本降九成
Deploying Multi-task Online Server with Large Language Model
- 分三步：筛选任务、高资源任务微调、全任务联合微调
- 在多个基准上性能接近单任务模型，推理开销减少90.9%
- 适合需要多任务部署且追求低成本的工业级应用
工业界中大量任务需在线部署。传统方法为每个任务单独训练模型，导致开发和扩展成本过高，尤其在大语言模型场景下更为突出。尽管多任务学习可通过参数共享降低成本，但在真实应用中往往难以超越单任务方法的表现。为此，我们提出一种面向大语言模型的三阶段多任务学习框架：首先进行任务过滤，接着在高资源任务上进行微调，最后对所有任务进行联合微调。我们在单任务与多任务设置下进行了全面实验。结果表明，该方法在不同基准上表现接近单任务方法，同时将计算开销降低高达90.9%。
原文摘要 · Abstract (English)
In the industry, numerous tasks are deployed online. Traditional approaches often tackle each task separately by its own network, which leads to excessive costs for developing and scaling models, especially in the context of large language models. Although multi-task methods can save costs through parameter sharing, they often struggle to outperform single-task methods in real-world applications. To tackle these challenges, we present a three-stage multi-task learning framework for large language models. It involves task filtering, followed by fine-tuning on high-resource tasks, and finally fine-tuning on all tasks. We conducted comprehensive experiments in single-task and multi-task settings. Our approach, exemplified on different benchmarks, demonstrates that it is able to achieve performance comparable to the single-task method while reducing up to 90.9\% of its overhead.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。