用大模型自动生成可视化脚本，让科研人员用自然语言直接探索数据。
VizGenie: Toward Self-Refining, Domain-Aware Workflows for Next-Generation Scientific Visualization
- 通过大模型动态生成VTK代码，自动扩展可视化能力
- 自然语言指令可直接生成颅骨等特征的可视化结果
- 支持交互式提问和溯源，适合需要反复调试的科研场景
我们提出VizGenie，一个自进化、领域感知的智能可视化框架，利用大语言模型（LLM）协调一系列领域专用与动态生成的模块。用户可通过预设工具访问阈值过滤、切片提取和统计分析等功能；对于超出基础能力的任务，VizGenie自主调用LLM生成可视化脚本（如VTK Python代码），并在通过后台自动化验证后无缝集成，持续增强系统适应性与鲁棒性。其独特之处在于直观的自然语言接口，支持‘可视化颅骨’等高层特征查询，系统借助微调后的视觉模型进行图像分析与视觉问答（VQA），精准理解指令，打通领域知识与技术实现的鸿沟。用户还可对生成的可视化结果进行交互式VQA查询，促进深度探索。可靠性与可复现性通过检索增强生成（RAG）机制保障，提供上下文驱动响应并保留完整溯源记录。在复杂体数据集上的评估表明，该框架显著降低迭代可视化任务的认知负担。通过整合精选领域工具与LLM驱动的灵活性，VizGenie不仅加速洞察生成，更建立可持续演进的可视化实践体系，平台会随用户交互动态学习，持续强化以特征为中心的探索能力与可复现研究支持。
原文摘要 · Abstract (English)
We present VizGenie, a self-improving, agentic framework that advances scientific visualization through large language model (LLM) by orchestrating of a collection of domain-specific and dynamically generated modules. Users initially access core functionalities--such as threshold-based filtering, slice extraction, and statistical analysis--through pre-existing tools. For tasks beyond this baseline, VizGenie autonomously employs LLMs to generate new visualization scripts (e.g., VTK Python code), expanding its capabilities on-demand. Each generated script undergoes automated backend validation and is seamlessly integrated upon successful testing, continuously enhancing the system's adaptability and robustness. A distinctive feature of VizGenie is its intuitive natural language interface, allowing users to issue high-level feature-based queries (e.g., ``visualize the skull"). The system leverages image-based analysis and visual question answering (VQA) via fine-tuned vision models to interpret these queries precisely, bridging domain expertise and technical implementation. Additionally, users can interactively query generated visualizations through VQA, facilitating deeper exploration. Reliability and reproducibility are further strengthened by Retrieval-Augmented Generation (RAG), providing context-driven responses while maintaining comprehensive provenance records. Evaluations on complex volumetric datasets demonstrate significant reductions in cognitive overhead for iterative visualization tasks. By integrating curated domain-specific tools with LLM-driven flexibility, VizGenie not only accelerates insight generation but also establishes a sustainable, continuously evolving visualization practice. The resulting platform dynamically learns from user interactions, consistently enhancing support for feature-centric exploration and reproducible research in scientific visualization.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。