用大模型指导体积渲染的函数优化,让用户意图更直观地转化为可视化结果。
IntuiTF: MLLM-Guided Transfer Function Optimization for Direct Volume Rendering
- 利用多模态大模型理解用户意图,引导函数空间探索
- 结合进化式搜索与智能评估,实现高效精准的参数优化
- 适合医学影像、科学可视化等需要交互设计的场景
直接体积渲染(DVR)是可视化体数据的基础技术,其中传输函数(TF)在提取有意义结构中起关键作用。然而,由于用户意图与TF参数空间之间存在语义鸿沟,设计有效的TF仍不直观。尽管已有多种TF优化方法,但现有方法仍面临两大挑战:搜索空间庞大和泛化能力有限。为此,我们提出IntuiTF,一个利用多模态大语言模型(MLLM)引导TF优化以对齐用户意图的新框架。该方法包含两个核心组件:(1) 基于进化的探索器,有效探索TF空间;(2) 基于MLLM的用户对齐评估器,提供可泛化的渲染质量反馈。两者共同构建了高效的试错-洞察-重规划范式。我们进一步拓展框架,实现交互式TF设计系统。通过三个案例研究展示框架的广泛适用性,并通过大量实验验证各组件有效性。建议读者查看案例、演示视频及源码:https://github.com/wyysteelhead/IntuiTF
原文摘要 · Abstract (English)
Direct volume rendering (DVR) is a fundamental technique for visualizing volumetric data, where transfer functions (TFs) play a crucial role in extracting meaningful structures. However, designing effective TFs remains unintuitive due to the semantic gap between user intent and TF parameter space. Although numerous TF optimization methods have been proposed to mitigate this issue, existing approaches still face two major challenges: the vast exploration space and limited generalizability. To address these issues, we propose IntuiTF, a novel framework that leverages Multimodal Large Language Models (MLLMs) to guide TF optimization in alignment with user intent. Specifically, our method consists of two key components: (1) an evolution-driven explorer for effective exploration of the TF space, and (2) an MLLM-guided human-aligned evaluator that provides generalizable visual feedback on rendering quality. The explorer and the evaluator together establish an efficient Trial-Insight-Replanning paradigm for TF space exploration. We further extend our framework with an interactive TF design system. We demonstrate the broad applicability of our framework through three case studies and validate the effectiveness of each component through extensive experiments. We strongly recommend readers check our cases, demo video, and source code at: https://github.com/wyysteelhead/IntuiTF
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。