arXiv:2505.07064cs.HCcs.AI2025-05被引 17

用大模型让ParaView能听懂人话,直接操作可视化工具

ParaView-MCP: An Autonomous Visualization Agent with Direct Tool Use

  • 用MCP协议连接大模型与ParaView,实现自然语言控制
  • 可看画面反馈并自动调参,支持从图例复现和目标闭环调整
  • 适合科研、工程人员快速上手复杂可视化,降低使用门槛

尽管功能强大,ParaView等工具学习成本高,阻碍了广泛使用。本文提出ParaView-MCP,一个融合多模态大语言模型(MLLM)与ParaView的自主智能代理,不仅降低入门门槛,还为工具提供智能决策支持。系统采用标准化的模型上下文协议(MCP),实现大模型与ParaView Python API的直接交互,支持用户通过自然语言和视觉输入与软件沟通。通过引入可视化反馈机制,使代理能观察视窗输出,从而实现从示例重现已知可视化、根据用户目标闭环调整参数,甚至跨工具协作。该范式有望彻底改变人机可视化交互方式,推动科研与工业界在可视化工具开发上的广泛应用。

原文摘要 · Abstract (English)

While powerful and well-established, tools like ParaView present a steep learning curve that discourages many potential users. This work introduces ParaView-MCP, an autonomous agent that integrates modern multimodal large language models (MLLMs) with ParaView to not only lower the barrier to entry but also augment ParaView with intelligent decision support. By leveraging the state-of-the-art reasoning, command execution, and vision capabilities of MLLMs, ParaView-MCP enables users to interact with ParaView through natural language and visual inputs. Specifically, our system adopted the Model Context Protocol (MCP) - a standardized interface for model-application communication - that facilitates direct interaction between MLLMs with ParaView's Python API to allow seamless information exchange between the user, the language model, and the visualization tool itself. Furthermore, by implementing a visual feedback mechanism that allows the agent to observe the viewport, we unlock a range of new capabilities, including recreating visualizations from examples, closed-loop visualization parameter updates based on user-defined goals, and even cross-application collaboration involving multiple tools. Broadly, we believe such an agent-driven visualization paradigm can profoundly change the way we interact with visualization tools. We expect a significant uptake in the development of such visualization tools, in both visualization research and industry.

可视化大模型智能代理交互

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。