通过逆向环境历史生成海量命令行任务，提升代码智能体能力
CLI-Gym: Scalable CLI Task Generation via Agentic Environment Inversion
- 用代理模拟环境历史，逆向推导出带错误的初始状态作为任务
- 构建1655个环境密集型任务，是同类最大数据集
- 新模型LiberCoder在终端基准上准确率提升21.1%，达46.1%
智能体编程需要有效与运行时环境（如命令行界面）交互，以完成依赖问题修复、系统故障处理等任务。但如何规模化获取此类环境密集型任务仍缺乏研究。本文基于Dockerfile与智能体任务的类比，提出利用代理模拟并探索环境历史，通过追踪健康环境的历史状态，逆向生成存在运行时故障的早期状态，进而打包错误状态与对应错误信息形成任务。该方法命名为CLI-Gym，共生成1,655个环境密集型任务，为目前最大规模同类数据集。同时，结合筛选后的成功轨迹微调模型LiberCoder，在Terminal-Bench上取得46.1%的准确率，相比基线提升21.1个百分点。这是首个公开的可扩展环境密集型任务生成流水线。
原文摘要 · Abstract (English)
Agentic coding requires agents to effectively interact with runtime environments, e.g., command line interfaces (CLI), so as to complete tasks like resolving dependency issues, fixing system problems, etc. But it remains underexplored how such environment-intensive tasks can be obtained at scale to enhance agents' capabilities. To address this, based on an analogy between the Dockerfile and the agentic task, we propose to employ agents to simulate and explore environment histories, guided by execution feedback. By tracing histories of a healthy environment, its state can be inverted to an earlier one with runtime failures, from which a task can be derived by packing the buggy state and the corresponding error messages. With our method, named CLI-Gym, a total of 1,655 environment-intensive tasks are derived, being the largest collection of its kind. Moreover, with curated successful trajectories, our fine-tuned model, named LiberCoder, achieves substantial absolute improvements of +21.1% (to 46.1%) on Terminal-Bench, outperforming various strong baselines. To our knowledge, this is the first public pipeline for scalable derivation of environment-intensive tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。