arXiv:2604.11465cs.AI2026-04

不加训练,仅靠推理时调度,让小模型性能翻倍。

Three Roles, One Model: Role Orchestration at Inference Time to Close the Performance Gap Between Small and Large Agents

论文配图:Three Roles, One Model: Role Orchestration at Inference Time to Close the Performance Gap Between Small and Large Agents
图 1 · 摘自论文原文
  • 同一模型分三角色:摘要、推理、纠错,协同工作
  • 8B模型在4比特下任务完成率从3.0%提升至5.9%
  • 无需新训练,小模型可超越4倍大的对手

大语言模型代理在真实工具使用任务中表现优异,但在资源受限设备上部署高能力代理仍具挑战。本文研究仅通过推理时结构化干预(无需额外训练算力)能否提升小模型在复杂多步环境中的表现。在单张24GB GPU上,我们评估了Qwen3-8B在AppWorld基准下的全精度(FP16)与4比特量化(AWQ)配置。未经干预时,原始模型任务目标完成率仅为5.4%(FP16)和3.0%(AWQ)。基于系统性失败模式分析,我们提出三阶推理支架管道:(1)摘要模型压缩对话历史并保留关键信息;(2)主代理在压缩上下文中推理;(3)独立校正模型无历史访问地审查并修正代码输出,打破重复错误循环。该方案应用于相同未修改模型后,任务完成率分别提升至8.9%(FP16)和5.9%(AWQ),性能近乎翻倍,尤其在难度1任务上从15.8%升至26.3%(FP16)、5.3%升至14.0%(AWQ)。全精度下,该支架化8B模型超越原版测试中DeepSeek-Coder 33B Instruct的7.1%,证明结构化推理干预可使小模型媲美四倍大小的系统。本方法被形式化为对冻结基础模型的支架策略,通过三次不同条件调用同一权重实现,关联到测试时计算扩展与强化学习中的动作空间塑造。

原文摘要 · Abstract (English)

Large language model (LLM) agents show promise on realistic tool-use tasks, but deploying capable agents on modest hardware remains challenging. We study whether inference-time scaffolding alone, without any additional training compute, can improve the performance of a small model in complex multi-step environments. Operating on a single 24GB GPU, we evaluate Qwen3-8B on the AppWorld benchmark under both full-precision and 4-bit quantized configurations. Without any intervention, the raw model achieves just 5.4% (FP16) and 3.0% (AWQ) task goal completion. Guided by a systematic failure mode analysis, we introduce a three-tier inference scaffolding pipeline that deploys the same frozen model in three distinct roles: (1) a summarization model that preserves critical artifacts (tokens, credentials, API responses) while compressing dialogue history; (2) the main agent model that reasons over the compressed context; and (3) an isolated correction model that reviews and revises the agent's code output without access to conversation history, breaking repetitive failure loops. Applied to the same unmodified model, this scaffolding yields 8.9% (FP16) and 5.9% (AWQ) task goal completion, roughly doubling performance in both settings, with particularly strong gains on difficulty-1 tasks (15.8% to 26.3% FP16; 5.3% to 14.0% AWQ). On full-precision inference, our scaffolded 8B model surpasses DeepSeek-Coder 33B Instruct (7.1%) from the original AppWorld evaluation, demonstrating that structured inference-time interventions can make small models competitive with systems 4 times their size. We formalize the approach as a scaffolded policy over a frozen base model, three invocations of the same weights with different conditioning, drawing connections to test-time compute scaling and action-space shaping in reinforcement learning.

小模型推理优化多角色协作性能提升

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。