动态调整智能体交互时长，提升长任务表现并节省计算资源。
Elastic Horizon: Discovering the Effective Interaction Frontier in Agentic Reinforcement Learning
- 用成功轨迹长度的90百分位动态监控最优交互边界。
- 在两个数据集上实现最高成功率，比固定时长方法减少25%的计算开销。
- 适合需要高效长程决策的大型语言模型应用。
扩大每回合环境交互次数可提升大语言模型智能体在长周期任务上的表现，基于课程的学习方法通过逐步增加交互时长优于固定时长设置。然而，现有方法为开环设计：时长单调递增直至人为设定上限，缺乏检测何时继续扩展不再有效的能力。本文提出有效交互前沿假设：存在一个动态边界，越过此边界后额外交互带来的收益递减，而成本线性增长。为此提出Elastic Horizon，一种闭环控制器，通过追踪成功轨迹长度的90百分位来定位该边界。在AppWorld和BFCL数据集上，固定时长实验显示明显的性能饱和平台；Elastic Horizon从不足或过量初始化均能将时长稳定在饱和区间内，对7B与14B参数量模型均取得最佳成功率，并减少最多25%的每步轨迹令牌消耗。本工作将研究范式从‘如何扩展交互时长’转向‘何时停止扩展’。
原文摘要 · Abstract (English)
Scaling the interaction horizon-the maximum number of environment interactions per episode-improves LLM agents on long-horizon tasks, and curriculum-based methods that progressively expand the horizon outperform fixed-horizon alternatives. However, existing schedules are open-loop: they monotonically increase the horizon until a manually specified maximum, with no mechanism to detect when further expansion stops helping. We propose the effective interaction frontier hypothesis: a dynamic boundary beyond which additional interactions yield diminishing returns while cost grows linearly. We then introduce Elastic Horizon, a closed-loop controller that tracks this boundary via the 90th percentile of successful trajectory lengths. On AppWorld and BFCL, fixed-horizon sweeps reveal clear saturation plateaus; Elastic Horizon stabilizes the horizon inside the saturation band from both under- and over-capacity initializations, attains the best success rates across 7B and 14B backbones, and saves up to 25% of per-step trajectory tokens. Our work shifts the paradigm from how to scale interaction horizons to when to stop scaling.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。