arXiv:2506.07976cs.LGcs.AI2025-06被引 40

让智能体在测试时通过持续互动来动态调整行为,提升网页任务完成率。

Thinking vs. Doing: Agents that Reason by Scaling Test-Time Interaction

  • 通过延长交互时间轴,让智能体在单次运行中自主探索与重规划。
  • 在WebVoyager和WebArena上达到开源模型最佳表现,成功率显著提升。
  • 适合需要自适应决策的智能体研发者,尤其关注在线学习与交互优化。

当前测试时扩展主要依赖生成长推理轨迹(‘思考’),但在需与环境交互的任务中,此法无法获取新信息或动态调整行为。本文提出测试时交互扩展(TTI),通过增加智能体的交互时长,支持探索、回溯与动态重规划等复杂行为。实验表明,仅靠提示工程即可在网页基准上实现非平凡的性能提升。在此基础上,我们提出一种基于课程的在线强化学习方法,动态调节采样长度。使用Gemma 3 12B模型,TTI在WebVoyager与WebArena上实现了当前最优的开源开放数据表现,并能自适应平衡探索与利用。结果证明,交互扩展是继每步计算量扩展之外的重要补充方向,为训练自适应智能体开辟新路径。

原文摘要 · Abstract (English)

The current paradigm of test-time scaling relies on generating long reasoning traces ("thinking" more) before producing a response. In agent problems that require interaction, this can be done by generating thinking traces before acting in the world. However, this process does not allow agents to acquire new information from the environment or adapt their behavior over time. In this work, we propose to scale test-time interaction, an untapped dimension of test-time scaling that increases the agent's interaction horizon to enable running rich behaviors such as exploration, backtracking, and dynamic re-planning within a single rollout. To demonstrate the promise of this scaling dimension, we study the domain of web agents. We first show that even prompting-based interaction scaling without any training can improve task success on web benchmarks non-trivially. Building on this, we introduce TTI (Test-Time Interaction), a curriculum-based online reinforcement learning (RL) approach that trains agents by adaptively adjusting their rollout lengths. Using a Gemma 3 12B model, TTI produces state-of-the-art open-source, open-data web agents on WebVoyager and WebArena benchmarks. We further show that TTI enables agents to balance exploration and exploitation adaptively. Our results establish interaction scaling as a powerful, complementary axis to scaling per-step compute, offering new avenues for training adaptive agents.

智能体交互扩展在线学习网页代理

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。