用大模型自动生成功能完整的地理空间仪表盘，提升决策效率。
Context-Aware Visual Prompting: Automating Geospatial Web Dashboards with Large Language Models and Agent Self-Validation for Decision Support
- 通过视觉提示提取界面语义，驱动代码生成。
- 自验证机制确保输出可靠性，支持多页交互界面。
- 适合需要快速构建地理分析系统的科研与政府机构。
开发基于网络的地理空间仪表盘用于风险分析和决策支持，常面临大规模多维环境数据可视化困难、实现复杂度高和自动化程度低的问题。本文提出一种生成式AI框架，利用大语言模型（LLMs）从用户定义的输入（如界面原型、需求说明和数据源）自动生成交互式地理空间仪表盘。通过引入结构化知识图谱，工作流将领域知识嵌入生成过程，实现准确且上下文感知的代码补全。关键创新在于上下文感知视觉提示（CAVP）机制，可从视觉布局中提取并编码界面语义，引导基于LLM的代码生成。框架还集成自验证机制，采用基于代理的LLM与Pass@k评估及语义指标相结合，确保输出可靠性。仪表盘片段与可视化代码库及本体表示相结合，形成可扩展的管道，使用MVVM架构模式生成基于React的完整代码。结果表明，该方法性能优于基线，功能超过第三方平台，支持多页、全功能界面。成功实现了基于LLM的自动化代码生成、部署，并验证了链式思维AI代理在自验证中的应用。该整合方法依托结构化知识与视觉提示，为增强风险分析与决策提供了创新解决方案。
原文摘要 · Abstract (English)
The development of web-based geospatial dashboards for risk analysis and decision support is often challenged by the difficulty in visualization of big, multi-dimensional environmental data, implementation complexity, and limited automation. We introduce a generative AI framework that harnesses Large Language Models (LLMs) to automate the creation of interactive geospatial dashboards from user-defined inputs including UI wireframes, requirements, and data sources. By incorporating a structured knowledge graph, the workflow embeds domain knowledge into the generation process and enable accurate and context-aware code completions. A key component of our approach is the Context-Aware Visual Prompting (CAVP) mechanism, which extracts encodes and interface semantics from visual layouts to guide LLM driven generation of codes. The new framework also integrates a self-validation mechanism that uses an agent-based LLM and Pass@k evaluation alongside semantic metrics to assure output reliability. Dashboard snippets are paired with data visualization codebases and ontological representations, enabling a pipeline that produces scalable React-based completions using the MVVM architectural pattern. Our results demonstrate improved performance over baseline approaches and expanded functionality over third party platforms, while incorporating multi-page, fully functional interfaces. We successfully developed a framework to implement LLMs, demonstrated the pipeline for automated code generation, deployment, and performed chain-of-thought AI agents in self-validation. This integrative approach is guided by structured knowledge and visual prompts, providing an innovative geospatial solution in enhancing risk analysis and decision making.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。