arXiv:2608.10333cs.LG2026-08

让小模型通过自我迭代变强,比单纯调用大模型更省成本。

MERA: Model Evolution and Routing with Skill Adaptation for Agentic Systems at Scale

  • 用失败任务重放生成教学数据,逐步构建可复用的技能库并微调小模型。
  • 4轮迭代后小模型在HumanEval上通过率从28.7%提升至49.7%,接近大模型表现。
  • 适合需要低成本部署高智能代理系统的团队,尤其关注持续进化能力。

LLM代理在单个任务中执行异构的模型调用序列:部分调用需深度推理,部分为格式化或工具参数构造等结构化步骤。现有路由方法通过将简单调用分配给便宜的小模型、复杂调用交给大模型来降低推理成本,但小模型能力不变,节省上限受限于其原有水平。MERA则主动提升小模型自身能力,以单次模型调用为适应单位。每轮循环中,MERA回放失败的学生调用,获取执行验证的教师示范,将重复出现的流程提炼为不断更新的SkillBook,并通过监督学习和可选的GRPO微调学生LoRA适配器。路由作为部署支持机制:改进后的小模型由成本校准的路由器调度,配备验证器回退,仅当联合回放保持任务质量时才接受候选的SkillBook、适配器或路由策略。实验显示,四轮适应使Qwen2.5-Coder-1.5B在独立测试集HumanEval+MBPP上的通过率从28.7%提升至49.7%。在验证器回退下,部署策略在仅60.8%的始终使用大模型(Luna)成本下仍保持88.3%的通过率。在TAU-2数据集上,微调后的Qwen3.5-2B从14/35提升至18/35,达到未适应的4B模型水平。结果表明,验证器支持的多轮自适应能真正提升小模型能力,而非仅依赖固定学生的路由策略。

原文摘要 · Abstract (English)

LLM agents execute heterogeneous sequences of model calls within a single task: some invocations require careful reasoning, while others are structured steps such as formatting or tool-argument construction. Prior routing methods exploit this asymmetry by assigning easy invocations to a cheaper small model and difficult ones to a large model. Such policies reduce inference cost, but they leave the small model's capability unchanged, so attainable savings remain bounded by the work the student can already solve. MERA instead improves the small model itself, using a single model invocation as the unit of adaptation. In each cycle, MERA replays failed student invocations to obtain execution-verified teacher demonstrations, distills recurring procedures into an iteratively updated SkillBook, and fine-tunes a student LoRA adapter via supervised learning and optional GRPO. Routing serves as supporting machinery for deployment: the improved student is served behind a cost-calibrated router with verifier-backed fallback, and a candidate SkillBook, adapter, or router is admitted only when joint replay preserves task quality. Empirically, four-cycle adaptation raises Qwen2.5-Coder-1.5B from 28.7% to 49.7% pass on held-out HumanEval+MBPP. Under verifier-backed fallback, the deployed policy retains 88.3% pass at 60.8% of always-Luna cost. On TAU-2, a fine-tuned Qwen3.5-2B improves from 14/35 to 18/35 and matches an unadapted 4B model. These results indicate that verifier-backed multi-cycle adaptation can increase small-model capability, rather than only routing around a fixed student.

代理系统小模型优化技能迁移成本控制

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。