调整工具调用框架能显著提升大模型代理的适应能力。
The Interplay of Harness Design and Post-Training in LLM Agents
- 将工具调用框架视为可优化变量,而非固定设计
- 框架适配后,模型在分布内和分布外任务上性能均提升
- 适合需要动态适应新工具环境的智能体研发者
集成工具的大语言模型代理通常依赖于一个‘框架’:即决定暴露哪些工具、如何描述它们以及每一步观察伴随何种辅助信息的支撑结构。尽管代理常进行后训练,但该框架通常被视为固定工程细节,设计仅限于无训练场景。此外,现有后训练算法假设环境静态,而实际部署中工具环境与任务常发生变化。为此,我们扩展了 $ exttt{ALFWorld}$,(i) 将框架视为可控设计维度,(ii) 支持在任务与工具环境变化下的评估。基于此,我们系统分析框架设计对后训练在分布内与分布外(OOD)设置下影响。实验表明,框架感知的后训练不仅提升分布内性能,还使代理稳健适应分布外情况。在设计简化的框架下,后训练在更强工具环境变化下性能急剧下降,进一步凸显在环境变化时框架感知后训练的重要性。
原文摘要 · Abstract (English)
Tool-integrated LLM agents are often wrapped within a harness: the scaffolding that determines which tools are exposed, how they are described, and what auxiliary information accompanies each per-step observation. While agents are routinely post-trained, this scaffolding is typically treated as a fixed engineering detail, with design effort limited to the training-free regime. Moreover, existing post-training algorithms assume a static environment, even though tool environments and tasks often shift upon deployment. To address this gap, we extend $\texttt{ALFWorld}$ (i) to treat the harness as a controllable design dimension and (ii) to support evaluation under task and tool environment shifts. Building on this, we systematically analyze how the harness design influences post-training in both in-distribution and out-of-distribution (OOD) settings. We empirically show that harness-aware post-training not only improves in-distribution performance but also enables agents to robustly adapt to OOD settings. Under a harness with minimal design effort, post-training suffers a drastic performance drop under stronger tool environment shifts, further highlighting the importance of harness-aware post-training under such shifts.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。