arXiv:2505.07214cs.HCcs.AI2025-05被引 6

用AI助手在VR中实现更直观的医学影像分割,提升医生操作效率。

Towards user-centered interactive medical image segmentation in VR with an assistive AI agent

  • 通过语音交互让AI助手协助定位、分割和可视化3D医学图像
  • 用户仅需几个点就能精修分割结果,系统支持真实尺度三维展示
  • 对比三种输入方式,证明头指和眼动更适合沉浸式交互场景

在疾病分析与手术规划中,手动分割体层医学影像(如MRI、CT)耗时费力且易出错,而全自动算法可借助用户反馈改进。结合最新放射科AI基础模型与虚拟现实(VR)的直观交互能力,我们提出SAMIRA——一种面向医疗VR的对话式AI助手,帮助用户定位、分割并可视化三维医学概念。通过语音交互,该助手可辅助理解影像特征、识别临床目标,并生成可仅用少量点提示即完成精修的分割掩码。系统还支持病灶的真实尺度三维可视化,增强个体化解剖认知。为探索在沉浸式人机协作流程中近远距离注意力切换下的最优交互范式,我们比较了VR控制器指针、头部指针和眼动追踪三种输入方式。用户研究显示,系统可用性评分高(SUS=90.0±9.0),整体任务负荷低,且对系统引导性、培训潜力及AI集成价值有强支持。

原文摘要 · Abstract (English)

Crucial in disease analysis and surgical planning, manual segmentation of volumetric medical scans (e.g. MRI, CT) is laborious, error-prone, and challenging to master, while fully automatic algorithms can benefit from user feedback. Therefore, with the complementary power of the latest radiological AI foundation models and virtual reality (VR)'s intuitive data interaction, we propose SAMIRA, a novel conversational AI agent for medical VR that assists users with localizing, segmenting, and visualizing 3D medical concepts. Through speech-based interaction, the agent helps users understand radiological features, locate clinical targets, and generate segmentation masks that can be refined with just a few point prompts. The system also supports true-to-scale 3D visualization of segmented pathology to enhance patient-specific anatomical understanding. Furthermore, to determine the optimal interaction paradigm under near-far attention-switching for refining segmentation masks in an immersive, human-in-the-loop workflow, we compare VR controller pointing, head pointing, and eye tracking as input modes. With a user study, evaluations demonstrated a high usability score (SUS=90.0 $\pm$ 9.0), low overall task load, as well as strong support for the proposed VR system's guidance, training potential, and integration of AI in radiological segmentation tasks.

医学影像VR交互AI助手分割

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。