用轻量缝合层把多个专家模型合并成一个通用大模型。
BTS: Harmonizing Specialized Experts into a Generalist LLM
- 通过插入轻量缝合层，将多个已训练专家模型与主模型连接。
- 在少量混合数据上训练缝合层，使主模型能泛化到新领域。
- 支持灵活增删专家，保持原有专长且性能优于其他合并方法。
我们提出分支-训练-缝合（BTS）算法，一种高效灵活的训练方法，可将独立训练的大语言模型专家整合为单一通用模型。基于Li等人的工作，从一个基础语言模型出发，通过持续预训练将其分支为特定领域（如编程或数学）的专家模型。BTS通过在冻结的专家模型与基础模型之间插入轻量级缝合层，并在少量专家领域混合数据上进行训练，实现专家融合。缝合层使基础模型在前向传播中能够整合任意数量专家的表示，从而在不改变原有模型的前提下实现对新领域的泛化能力。由于不修改原有模型，BTS具备模块化和灵活性：专家可轻松移除，新专家仅需少量训练即可加入。相比其他模型融合方法，BTS在多种下游任务中表现最优，同时保留各专家的专长能力。
原文摘要 · Abstract (English)
We present Branch-Train-Stitch (BTS), an efficient and flexible training algorithm for combining independently trained large language model (LLM) experts into a single, capable generalist model. Following Li et al., we start with a single seed language model which is branched into domain-specific (e.g., coding or math) experts with continual pretraining. BTS combines experts into a generalist model using lightweight stitch layers, which are inserted between frozen experts and the seed LLM, and trained on a small datamix of the expert domains. Stitch layers enable the seed LLM to integrate representations from any number of experts during the forward pass, allowing it to generalize to new domains, despite remaining frozen. Because BTS does not alter the constituent LLMs, BTS provides a modular and flexible approach: experts can be easily removed and new experts can be added with only a small amount of training. Compared to alternative model merging approaches, BTS yields the best generalist performance on a variety of downstream tasks, retaining the specialized capabilities of each of the experts.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。