小模型自进化框架让低算力环境下的智能体自动优化推理策略。
PACE: Two-Timescale Self-Evolution for Small Language Model Agents

- 分时迭代优化:先调提示词,再谨慎更新控制逻辑。
- 在12组实验中最高提升9.2%,优于纯提示优化和单模式进化。
- 适合资源受限场景,无需大模型指导,可自主验证改进方案。
将语言模型智能体部署于生产环境通常需要大量计算资源和人工调参,包括提示词、解析器、验证器等组件。自进化提供了一种替代方案,但现有框架多依赖前沿大模型来诊断错误、提出修改并评估更新。本文研究在资源受限下,冻结的小语言模型(SLMs)能否作为有效的自进化智能体。提出PACE(Prompt And Control Logic Evolution)框架,通过双时间尺度协调低风险提示词优化与高风险控制逻辑更新:在固定控制逻辑下持续优化提示词直至收益饱和,随后在保留验证集上筛选受控的逻辑更新。在3个参数量从4B到14B的冻结SLM骨干和4个受控基准测试中,PACE在全部12个组合上表现最佳,相较基础SLM智能体最高提升9.2%相对性能,相较更强的单模式进化基线最高提升5.4%。τ-基准案例研究进一步显示,相比基础版本和仅优化提示词的方案,PACE显著提升多轮工具使用成功率。结果表明,在不更新模型权重或依赖前沿模型教师的前提下,可靠的SLM智能体自进化是可行的,其核心优势并非单一最优解法,而是自主、验证驱动的任务适配推理策略发现。
原文摘要 · Abstract (English)
Deploying language-model agents in production often requires substantial compute and human effort to tune prompts, parsers, validators, and other components of the agent pipeline. Self-evolution offers a promising alternative, but most existing frameworks assume access to frontier models that can reliably diagnose failures, propose revisions, and judge their own updates. We study whether frozen small language models (SLMs) can serve as effective self-evolving agents under resource constraints. We propose PACE (Prompt And Control Logic Evolution), a two-timescale framework that coordinates low-risk prompt refinement with higher-risk control-logic updates. PACE evolves prompts under fixed control logic until prompt-level gains saturate, then considers constrained control-logic updates that are accepted through held-out validation. Across three frozen SLM backbones ranging from 4B to 14B parameters and four controlled benchmarks, PACE achieves the best performance on all 12 backbone--benchmark combinations, improving over vanilla SLM agents by up to +9.2% relative improvement and over the stronger single-mode evolution baseline by up to +5.4% relative improvement. A tau-bench case study further shows that PACE improves multi-turn tool-use success over vanilla and prompt-only evolution. These results suggest that reliable SLM agent self-evolution is possible without updating model weights or relying on frontier-model teachers, and that the key benefit is not any single final solver pattern but autonomous, validated discovery of task-appropriate inference strategies.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。