系统梳理终端环境中的AI代理能力与评估方法
Terminal Agents: A Survey of AI Agents in Command-Line Environments

- 以终端命令执行为核心,构建七维能力框架统一分析代理行为
- 实验证明评估结果受系统环境和运行条件显著影响
- 强调需记录可复现的操作轨迹以支持过程质量评估
大语言模型代理越来越多地通过终端环境执行任务,但现有综述将终端交互分散于软件工程、工具使用与计算机行为研究中。本文将终端代理定义为以终端命令执行、文本反馈和状态化环境交互为核心推进机制的系统。基于终端执行这一组织视角,本综述建立工作负载边界,通过七维终端能力图谱连接系统架构、能力获取与评估方法。分析表明,实际行为由模型、接口、工具链、运行时及环境共同决定。可执行路径使学习扎根于行动后果、验证与恢复机制,而当前评估多关注最终结果,对过程质量、恢复能力与治理水平覆盖不均。固定条件下的诊断实验揭示:基准家族呈现不同过程信号,匹配系统对比暴露基准依赖性能与组件归因局限。这些发现推动显式报告系统与运行时条件,并依托可回放轨迹与过程级证据。该框架为跨软件工程与新兴应用领域研究终端代理提供统一基础。
原文摘要 · Abstract (English)
Large language model agents increasingly act through terminals, yet existing surveys disperse terminal-mediated behavior across software engineering, tool use, and computer-use research. We regard terminal agents as systems whose dominant progress-bearing action--observation loop is mediated by terminal command execution, textual feedback, and stateful environment interaction. Using terminal-mediated execution as an organizing lens, this survey establishes workload-level boundaries and connects system architecture, competence acquisition, and evaluation through a seven-dimensional terminal competence profile. Our synthesis shows that realized behavior is jointly shaped by the model, interface, harness, runtime, and environment. Executable trajectories ground learning in action consequences, verification, and recovery, whereas prevailing evaluations emphasize final outcomes and expose process quality, recovery, and governance unevenly. Bounded fixed-condition diagnostics illustrate two implications: benchmark families expose different process signals, and matched system comparisons reveal benchmark-dependent performance and limits of component attribution. These findings motivate explicit reporting of system and runtime conditions, supported by replayable traces and process-level evidence. The framework provides a unified basis for studying terminal-mediated agency across software engineering and emerging application domains.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。