打造智能虚拟超声医师,打通医生、机器人与患者沟通屏障
Intelligent Virtual Sonographer (IVS): Enhancing Physician-Robot-Patient Communication
- 用大模型构建XR虚拟助手,实现三方实时对话
- 医生指令由机器人执行,患者同步获知操作进展
- 兼顾专业沟通与情感安抚,提升医患信任和体验
大型语言模型(LLM)与机器人技术的进步为人机交互开辟了新可能,尤其在机器人超声领域。现有研究多聚焦于患者-机器人或医生-机器人互动,而连接三者沟通的智能虚拟超声医师(IVS)仍缺乏探索。本文提出一种基于扩展现实(XR)的对话式虚拟代理,实现医生、机器人超声系统(RUS)与患者之间的实时交互。该代理以专业方式与医生沟通,同时向患者提供共情解释与安慰,并主动执行医生指令,透明化地向患者传达操作过程。系统融合LLM对话能力、语音识别、文本转语音与机器人控制,显著提升了机器人超声采集的效率、清晰度与可及性。本工作为理解IVS如何弥合医-机-患沟通鸿沟迈出第一步,增强了医生对机器人的控制感与信任,同时改善患者体验与接受度。
原文摘要 · Abstract (English)
The advancement and maturity of large language models (LLMs) and robotics have unlocked vast potential for human-computer interaction, particularly in the field of robotic ultrasound. While existing research primarily focuses on either patient-robot or physician-robot interaction, the role of an intelligent virtual sonographer (IVS) bridging physician-robot-patient communication remains underexplored. This work introduces a conversational virtual agent in Extended Reality (XR) that facilitates real-time interaction between physicians, a robotic ultrasound system(RUS), and patients. The IVS agent communicates with physicians in a professional manner while offering empathetic explanations and reassurance to patients. Furthermore, it actively controls the RUS by executing physician commands and transparently relays these actions to the patient. By integrating LLM-powered dialogue with speech-to-text, text-to-speech, and robotic control, our system enhances the efficiency, clarity, and accessibility of robotic ultrasound acquisition. This work constitutes a first step toward understanding how IVS can bridge communication gaps in physician-robot-patient interaction, providing more control and therefore trust into physician-robot interaction while improving patient experience and acceptance of robotic ultrasound.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。