对比不同AI助手在科学绘图任务中的表现,发现各有优劣。
Exploring LLM Agent Designs and Interaction Modalities for Scientific Visualization

- 用三类智能体分别处理绘图任务,比较效率与稳定性。
- 通用编码助手成功率最高但耗能大,专用助手更省资源。
- 持续记忆能提升重复任务表现,依赖交互方式和反馈质量。
本文研究大型语言模型(LLM)代理在科学可视化(SciVis)任务中的表现,这些任务需从自然语言指令生成可视化工作流。我们在15个基准任务上比较了三类代表性代理设计:具备结构化工具使用的领域专用代理、计算机操作代理和通用编程代理,评估了可视化质量、效率、鲁棒性、计算成本及持久记忆的影响。同时研究了代码脚本、模型上下文协议(MCP)或API调用、命令行接口(CLI)和图形用户界面(GUI)等交互模式。结果揭示了不同代理设计与交互模式之间的明显权衡:通用编程代理任务成功率最高但计算开销大,领域专用代理更高效稳定但灵活性不足;计算机操作代理在单步操作中表现良好,但在多步骤工作流中表现不佳。在CLI和GUI两种环境下,持久记忆均提升了重复试验的性能,但其有效性取决于交互模式和反馈质量。研究建议未来科学可视化系统应结合结构化工具使用、交互能力与自适应记忆机制,以平衡性能、鲁棒性与灵活性。
原文摘要 · Abstract (English)
This paper examines how large language model (LLM) agents perform on scientific visualization (SciVis) tasks that require generating visualization workflows from natural-language instructions. We compare three representative agent designs: domain-specific agents with structured tool use, computer-use agents, and general-purpose coding agents, across 15 benchmark tasks, evaluating visualization quality, efficiency, robustness, computational cost, and the impact of persistent memory. We further study interaction modalities, including code scripts, model context protocol (MCP) or API calls, command-line interfaces (CLI), and graphical user interfaces (GUI). Our goal is to characterize the tradeoffs among representative SciVis agent configurations used in practice. The results reveal clear tradeoffs across agent designs and interaction modalities. General-purpose coding agents achieve the highest task success rates but incur greater computational cost, whereas domain-specific agents are more efficient and stable but less flexible. Computer-use agents perform well on individual operations but struggle with multi-step workflows. Across both CLI- and GUI-based settings, persistent memory improves performance over repeated trials, but its effectiveness depends on the interaction mode and the quality of feedback. These findings suggest that future SciVis systems should combine structured tool use, interactive capabilities, and adaptive memory mechanisms to balance performance, robustness, and flexibility.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。