用自然语言指导3D体数据视角导航，自动找最相关视图。
Natural Language-Driven Viewpoint Navigation for Volume Exploration via Semantic Block Representation
- 用语义块编码体数据结构，结合CLIP分数提供语义引导
- 强化学习框架根据用户意图高效搜索匹配视图
- 适合科研人员快速理解复杂科学数据，无需3D操作经验
体数据探索对科学数据解读至关重要。然而，对于缺乏领域知识或3D导航经验的用户而言，选择最佳视角仍具挑战性。本文提出一种基于自然语言交互的新型框架，通过编码体数据块来捕捉并区分潜在结构，并引入CLIP Score机制为块注入语义信息以指导导航。导航由强化学习框架驱动，利用这些语义线索高效搜索并识别符合用户意图的视图。所选视图通过CLIP Score评估，确保其最能反映用户查询。该方法通过自动化视角选择，提升了体数据导航效率，增强了复杂科学现象的可解释性。
原文摘要 · Abstract (English)
Exploring volumetric data is crucial for interpreting scientific datasets. However, selecting optimal viewpoints for effective navigation can be challenging, particularly for users without extensive domain expertise or familiarity with 3D navigation. In this paper, we propose a novel framework that leverages natural language interaction to enhance volumetric data exploration. Our approach encodes volumetric blocks to capture and differentiate underlying structures. It further incorporates a CLIP Score mechanism, which provides semantic information to the blocks to guide navigation. The navigation is empowered by a reinforcement learning framework that leverage these semantic cues to efficiently search for and identify desired viewpoints that align with the user's intent. The selected viewpoints are evaluated using CLIP Score to ensure that they best reflect the user queries. By automating viewpoint selection, our method improves the efficiency of volumetric data navigation and enhances the interpretability of complex scientific phenomena.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。