用动态选专家+合并的方式，让大模型高效适配新领域实体识别。
Selecting and Merging: Towards Adaptable and Scalable Named Entity Recognition with Large Language Models
- 按领域相似度和样本表现选预训练专家模型
- 推理时合并专家，平均比统一模型高10%准确率
- 无需重训即可扩展，适合快速部署到新领域
监督微调（SFT）广泛用于对齐大语言模型（LLM）与信息抽取任务，如命名实体识别（NER）。然而，标注细粒度标签并训练特定领域模型成本高昂。现有方法通常在多个领域上训练统一模型，但缺乏适应性与可扩展性，因为并非所有训练数据都对目标领域有益，且模型扩容困难。本文提出SaM框架，在推理时动态选择并合并专家模型。针对目标领域，基于（i）领域相似度和（ii）采样实例的表现，从已有领域的预训练专家中选取。随后将这些专家合并，生成针对目标领域的优化模型。通过仅选择对目标域有益的专家进行合并，提升跨领域泛化能力，且无需额外训练。此外，专家可灵活增删，具备良好可扩展性。在多个基准上的实验证明，该框架平均优于统一模型10%。我们还提供了改进方向、实践经验和框架扩展建议。
原文摘要 · Abstract (English)
Supervised fine-tuning (SFT) is widely used to align large language models (LLMs) with information extraction (IE) tasks, such as named entity recognition (NER). However, annotating such fine-grained labels and training domain-specific models is costly. Existing works typically train a unified model across multiple domains, but such approaches lack adaptation and scalability since not all training data benefits target domains and scaling trained models remains challenging. We propose the SaM framework, which dynamically Selects and Merges expert models at inference time. Specifically, for a target domain, we select domain-specific experts pre-trained on existing domains based on (i) domain similarity to the target domain and (ii) performance on sampled instances, respectively. The experts are then merged to create task-specific models optimized for the target domain. By dynamically merging experts beneficial to target domains, we improve generalization across various domains without extra training. Additionally, experts can be added or removed conveniently, leading to great scalability. Extensive experiments on multiple benchmarks demonstrate our framework's effectiveness, which outperforms the unified model by an average of 10%. We further provide insights into potential improvements, practical experience, and extensions of our framework.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。