arXiv:2504.08696cs.SEcs.LG2025-04被引 6

为代码生成智能体设计可视化工具,加速问题诊断与实验分析。

SeaView: Software Engineering Agent Visual Interface for Enhanced Workflow

  • 通过轨迹可视化对比不同超参数或大模型的实验表现
  • 支持超过128k长度的交互轨迹分析,解决长序列难题
  • 适合研究者快速定位智能体错误,提升调试效率

基于自回归大语言模型的软件工程智能体(SWE agents)在真实编码任务中取得显著进展(在SWE-Bench Verified上超过60%),通过推理、环境交互和自我反思生成复杂操作轨迹。然而,这些轨迹往往超过大模型上下文长度(有时超过128k),且涉及长时间的智能体-环境交互,导致错误难以定位与理解,实验改进或退化也难追踪。尽管研究多聚焦于提升智能体性能,但对其输出的分析与可视化工具仍严重不足。为此,我们提出SeaView:一个面向软件工程智能体的可视化界面,旨在帮助研究人员高效分析实验。SeaView通过创新机制支持不同超参数或大模型实验的快速对比,帮助快速识别大模型或环境相关问题。用户研究表明,经验丰富的研究者可节省10至30分钟信息收集时间,新手则可减少30分钟至1小时的诊断耗时。

原文摘要 · Abstract (English)

Auto-regressive LLM-based software engineering (SWE) agents, henceforth SWE agents, have made tremendous progress (>60% on SWE-Bench Verified) on real-world coding challenges including GitHub issue resolution. SWE agents use a combination of reasoning, environment interaction and self-reflection to resolve issues thereby generating "trajectories". Analysis of SWE agent trajectories is difficult, not only as they exceed LLM sequence length (sometimes, greater than 128k) but also because it involves a relatively prolonged interaction between an LLM and the environment managed by the agent. In case of an agent error, it can be hard to decipher, locate and understand its scope. Similarly, it can be hard to track improvements or regression over multiple runs or experiments. While a lot of research has gone into making these SWE agents reach state-of-the-art, much less focus has been put into creating tools to help analyze and visualize agent output. We propose a novel tool called SeaView: Software Engineering Agent Visual Interface for Enhanced Workflow, with a vision to assist SWE-agent researchers to visualize and inspect their experiments. SeaView's novel mechanisms help compare experimental runs with varying hyper-parameters or LLMs, and quickly get an understanding of LLM or environment related problems. Based on our user study, experienced researchers spend between 10 and 30 minutes to gather the information provided by SeaView, while researchers with little experience can spend between 30 minutes to 1 hour to diagnose their experiment.

智能体可视化代码生成

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。