用模块化技能包让小模型高效实现多领域专精。
Skill Weaving: Efficient LLM Improvement via Modular Skillpacks
- 将大模型能力拆解为轻量领域技能包，按需加载。
- 9B模型在多任务上超越32B单体模型，速度提升4倍。
- 适合资源受限下需要多领域智能的应用场景。
大型语言模型需在多个领域具备专长，但现有方法难以在多领域能力与严格的内存及推理约束间取得平衡。本文提出SkillWeave，一种模块化改进框架，使语言模型在固定内存预算下实现领域专精。SkillWeave将通用模型的完整能力分解为轻量级、领域特定的技能包——即微调增量模块，重新组织并优化模型内部知识。为实现高效部署，引入SkillZip技术，将技能包压缩为紧凑且可直接推理的格式，支持低延迟执行下的强多领域表现。在多任务与代理型基准测试中，9B规模的SkillWeave模型优于多个基线，并超越32B单体模型，同时实现最高4倍的速度提升。
原文摘要 · Abstract (English)
Large language models increasingly require specialization across diverse domains, yet existing approaches struggle to balance multi-domain capacities with strict memory and inference constraints. In this work, we introduce SkillWeave, a modular improvement framework that enables LLMs to specialize under fixed memory budgets. SkillWeave partitions full capabilities of a general-purpose model into skillpacks -- lightweight, domain-specific delta modules -- that reorganize and refine the model's internal knowledge. For efficient deployment, SkillWeave integrates SkillZip to compress skillpacks into compact and inference-ready format, enabling strong multi-domain performance with low-latency execution. On multi-task and agentic benchmarks, a 9B SkillWeave model outperforms several baselines and even surpasses a 32B monolithic LLM, while achieving up to 4x speedup.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。