无需标记即可精确定位被遮盖的手术机器人,提升安全性和智能化水平。
Localising under the drape: proprioception in the era of distributed surgical robotic system
- 基于轻量立体相机与Transformer模型,实现无标记自主定位。
- 在140万张图像数据上训练,跟踪精度提升25%,抗遮挡能力更强。
- 适合追求高安全、低部署复杂度的智能手术系统研发人员。
尽管手术机器人具备高度机械精密性,但缺乏环境空间感知能力,易导致碰撞、系统恢复和流程中断,未来分布式多臂机器人系统将加剧这一问题。现有追踪系统依赖笨重的红外摄像头和反光标记,视野有限且增加术中硬件负担。本文提出一种无标记本体感知方法,可在机器人完全被无菌布覆盖的情况下实现精确定位。该方法仅使用轻量级双目RGB相机与新型Transformer深度学习模型,基于迄今最大的多中心空间手术机器人数据集(140万张来自人尸体及临床前活体研究的自标注图像)进行训练。通过追踪整个机器人及手术场景而非单个标记点,实现对遮挡具有鲁棒性的全局视图,支持手术场景理解与上下文感知控制。我们展示了在活体呼吸补偿中的临床潜力,可获取传统追踪无法观测的组织动态信息,并在多机器人系统中实现精准定位,为未来智能交互奠定基础。相比现有系统,本方法消除标记使用,提升追踪可见性25%。据我们所知,这是首个实现完全遮盖下手术机器人无标记本体感知的成果,显著降低部署复杂度,增强安全性,推动模块化与自主手术机器人的发展。
原文摘要 · Abstract (English)
Despite their mechanical sophistication, surgical robots remain blind to their surroundings. This lack of spatial awareness causes collisions, system recoveries, and workflow disruptions, issues that will intensify with the introduction of distributed robots with independent interacting arms. Existing tracking systems rely on bulky infrared cameras and reflective markers, providing only limited views of the surgical scene and adding hardware burden in crowded operating rooms. We present a marker-free proprioception method that enables precise localisation of surgical robots under their sterile draping despite associated obstruction of visual cues. Our method solely relies on lightweight stereo-RGB cameras and novel transformer-based deep learning models. It builds on the largest multi-centre spatial robotic surgery dataset to date (1.4M self-annotated images from human cadaveric and preclinical in vivo studies). By tracking the entire robot and surgical scene, rather than individual markers, our approach provides a holistic view robust to occlusions, supporting surgical scene understanding and context-aware control. We demonstrate an example of potential clinical benefits during in vivo breathing compensation with access to tissue dynamics, unobservable under state of the art tracking, and accurately locate in multi-robot systems for future intelligent interaction. In addition, and compared with existing systems, our method eliminates markers and improves tracking visibility by 25%. To our knowledge, this is the first demonstration of marker-free proprioception for fully draped surgical robots, reducing setup complexity, enhancing safety, and paving the way toward modular and autonomous robotic surgery.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。