用群体智能让大模型协作进化，无需调参也能高效适配新任务。
Model Swarms: Collaborative Search to Adapt LLM Experts via Swarm Intelligence
- 多大模型协同在权重空间搜索，通过集体智慧优化目标函数。
- 仅需200样本即可有效适配，相比基线最高提升21.0%。
- 适合需要快速适配、无标注数据少的场景，如个性化应用。
我们提出 Model Swarms，一种基于群体智能的协同搜索算法，用于无需调参地适配大语言模型（LLM）。该方法从一组 LLM 专家和一个效用函数出发，由表现最佳的检查点引导，多个模型在权重空间中协同移动并优化目标函数，实现模型适应。相比现有模型组合方法，Model Swarms 不依赖特定专家假设或组合方式，可在低数据场景（最少200个样本）下工作。大量实验表明，该方法能灵活适配单一任务、多任务领域、奖励模型及多样人类兴趣，在各类任务与上下文中优于超过12种基线方法，性能提升最高达21.0%。进一步分析发现，初始检查点中的模型可发现未预见能力，且协同搜索过程促成弱模型向强模型的转变。
原文摘要 · Abstract (English)
We propose Model Swarms, a collaborative search algorithm to adapt LLMs via swarm intelligence, the collective behavior guiding individual systems. Specifically, Model Swarms starts with a pool of LLM experts and a utility function. Guided by the best-found checkpoints across models, diverse LLM experts collaboratively move in the weight space and optimize a utility function representing model adaptation objectives. Compared to existing model composition approaches, Model Swarms offers tuning-free model adaptation, works in low-data regimes with as few as 200 examples, and does not require assumptions about specific experts in the swarm or how they should be composed. Extensive experiments demonstrate that Model Swarms could flexibly adapt LLM experts to a single task, multi-task domains, reward models, as well as diverse human interests, improving over 12 model composition baselines by up to 21.0% across tasks and contexts. Further analysis reveals that LLM experts discover previously unseen capabilities in initial checkpoints and that Model Swarms enable the weak-to-strong transition of experts through the collaborative search process.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。