arXiv:2608.00181cs.SEcs.AI2026-08中稿 · COLM

通过跨基准训练,让长时序智能体学会可迁移的工作方式。

Cross-Benchmark Generalization in Long-Horizon Agents

  • 用363个任务的MCP数据集,分两阶段微调大模型,不依赖外部评分或任务。
  • 在5个外部评测中表现提升,软件类任务也显著进步,证明能力可迁移。
  • 发现四类通用行为模式,适用于办公与编程等不同场景。

在自包含环境中的强化学习中,策略可能依赖环境特有规律(如工具模板、评分解析、任务结构)获取奖励,而非习得可迁移技能,而分布内留出测试集也共享这些规律。我们认为关键在于行为差异:训练后智能体如何行动,跨基准迁移才是检验其真正能力的正确路径。我们对Qwen3.5-122B-A10B这一开放权重专家混合模型,在27个类别共363个长时序MCP任务上进行后训练,采用SFT-然后强化学习的两阶段流程。工具竞技场表现用于初始基线和SFT教师选择,但训练过程中未引入任何外部基准任务、评分器、奖励信号、超参数或检查点选择依据。在贪婪解码的pass@1下,模型在五个外部评测中均优于基线:工具竞技场(+9.6个百分点)、τ²-基准(+5.3)、BFCL-V4(+3.5)、SWE-Bench Pro(+5.8)和终端基准2(+2.8)。尽管训练数据不含软件工程任务,两个软件工程基准仍取得提升。探索性成对轨迹分析识别出四类反复出现的行为差异:更谨慎地形成局部目标、构建与目标相关的工作状态、在局部修复中保持父目标稳定、验证完成情况。这些结果提供了描述性证据,表明长时序多工具后训练能够改变可跨领域迁移的工作方式。

原文摘要 · Abstract (English)

For reinforcement learning (RL) in self-contained environments, a policy can get rewards by exploiting environment-specific regularities (tool schemas, grader parsing, task templates) rather than by acquiring transferable skill, and an in-distribution holdout shares those regularities. We argue that the discriminating question is behavioral, namely how a trained agent acts, and that cross-benchmark transfer is the right place to look for it. We post-train an open-weight mixture-of-experts model (Qwen3.5-122B-A10B) on 363 long-horizon Model Context Protocol (MCP) tasks across 27 categories, using a two-stage SFT-then-RL pipeline. Toolathlon performance informed the initial base-family and SFT-teacher choices, but no external-benchmark task or grader entered training and no external score informed the reward, training hyperparameters, trained-checkpoint selection, or stopping. At greedy pass@1, the trained model improves over the base on five reported external evaluations: Toolathlon (+9.6 pp), $τ^2$-Bench (+5.3 pp), BFCL-V4 (+3.5 pp), SWE-Bench Pro (+5.8 pp), and Terminal-Bench 2 (+2.8 pp). Both software-engineering benchmarks improve despite the training collection containing no software-engineering tasks. An exploratory paired-trajectory analysis identifies four recurring behavioral differences (more careful local-goal formation, building goal-relevant working state, keeping parent goals stable through local repairs, and verifying completion) that appear in analogous forms across office workflows and code. These results provide descriptive evidence that long-horizon multi-tool post-training can change ways of working that transfer beyond its training domain.

强化学习跨域迁移长时序任务行为分析

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。