用专家稀疏反馈提升机器人超声学习效率，显著加快高质量成像获取。
Coaching a Robotic Sonographer: Learning Robotic Ultrasound with Sparse Expert's Feedback
- 结合强化学习与专家指导，通过稀疏反馈动态优化操作策略。
- 在模拟体模上使高质量图像获取量提升74.5%，学习速度加快25%。
- 适合需要降低操作门槛的医疗机器人研发与临床培训场景。
超声检查因其无创、无辐射、实时成像等优势被广泛应用于临床诊断与介入治疗，但其操作高度依赖专业人员的经验与训练。机器人超声（RUS）有望缓解这一困境，但实现人类水平的操作仍具挑战。现有基于演示学习（LfD）的方法依赖离线示范数据建模专家经验，但缺乏专家在训练过程中的主动参与。本文提出一种新型教练框架，将深度强化学习（DRL）与专家稀疏反馈相结合，利用自监督练习与专家修正共同优化策略。采用离策略Soft Actor-Critic（SAC）网络，奖励基于图像质量评分。专家反馈建模为部分可观测马尔可夫决策过程（POMDP），根据专家纠正动态更新策略参数。在体模验证中，该框架使学习速率提升25%，高质量图像获取量增加74.5%。
原文摘要 · Abstract (English)
Ultrasound is widely employed for clinical intervention and diagnosis, due to its advantages of offering non-invasive, radiation-free, and real-time imaging. However, the accessibility of this dexterous procedure is limited due to the substantial training and expertise required of operators. The robotic ultrasound (RUS) offers a viable solution to address this limitation; nonetheless, achieving human-level proficiency remains challenging. Learning from demonstrations (LfD) methods have been explored in RUS, which learns the policy prior from a dataset of offline demonstrations to encode the mental model of the expert sonographer. However, active engagement of experts, i.e. Coaching, during the training of RUS has not been explored thus far. Coaching is known for enhancing efficiency and performance in human training. This paper proposes a coaching framework for RUS to amplify its performance. The framework combines DRL (self-supervised practice) with sparse expert's feedback through coaching. The DRL employs an off-policy Soft Actor-Critic (SAC) network, with a reward based on image quality rating. The coaching by experts is modeled as a Partially Observable Markov Decision Process (POMDP), which updates the policy parameters based on the correction by the expert. The validation study on phantoms showed that coaching increases the learning rate by $25\%$ and the number of high-quality image acquisition by $74.5\%$.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。