用大模型让机器人自主做超声检查，解决医生短缺问题
USPilot: An Embodied Robotic Assistant Ultrasound System with Large Language Model Enhanced Graph Planner
- 用大模型+图神经网络规划超声操作流程
- 在公开数据集上任务规划准确率显著提升
- 可理解患者问题并自动执行检查，适合医疗辅助场景
在大语言模型时代，具身人工智能为机器人操作任务带来变革性机遇。超声成像是一种广泛应用且成本较低的医学诊断手段，但全球专业超声技师短缺导致其应用受限。为此，我们提出USPilot——一种基于大模型框架的具身式机器人超声助手系统，实现超声采集的自主化。USPilot可作为虚拟超声师，响应患者关于超声的问题，并根据用户意图执行超声扫描。通过微调大模型，USPilot展现出对超声相关问题与任务的深度理解。此外，系统引入大模型增强的图神经网络（GNN）来管理超声机器人API并充当任务规划器。实验表明，该方法在公开数据集上的任务规划准确率达到前所未有的水平。系统还展示了在自主理解与执行超声检查方面的巨大潜力。这些进展推动了无人化、自主化机器人超声系统的实现，有助于缓解医疗影像资源短缺问题。
原文摘要 · Abstract (English)
In the era of Large Language Models (LLMs), embodied artificial intelligence presents transformative opportunities for robotic manipulation tasks. Ultrasound imaging, a widely used and cost-effective medical diagnostic procedure, faces challenges due to the global shortage of professional sonographers. To address this issue, we propose USPilot, an embodied robotic assistant ultrasound system powered by an LLM-based framework to enable autonomous ultrasound acquisition. USPilot is designed to function as a virtual sonographer, capable of responding to patients' ultrasound-related queries and performing ultrasound scans based on user intent. By fine-tuning the LLM, USPilot demonstrates a deep understanding of ultrasound-specific questions and tasks. Furthermore, USPilot incorporates an LLM-enhanced Graph Neural Network (GNN) to manage ultrasound robotic APIs and serve as a task planner. Experimental results show that the LLM-enhanced GNN achieves unprecedented accuracy in task planning on public datasets. Additionally, the system demonstrates significant potential in autonomously understanding and executing ultrasound procedures. These advancements bring us closer to achieving autonomous and potentially unmanned robotic ultrasound systems, addressing critical resource gaps in medical imaging.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。