ROOM模拟肺部支气管镜检查,生成逼真的医疗训练数据。
ROOM: A Physics-Based Continuum Robot Simulator for Photorealistic Medical Datasets Generation
- 基于患者CT扫描,合成多模态医学影像数据。
- 生成带真实噪声与光照的RGB图、深度图等,支持机器人算法训练。
- 适用于支气管镜导航、姿态估计等任务,适合医学机器人研究者。
连续体机器人正推动支气管镜手术发展,可进入复杂肺部气道并实现精准干预。然而其开发受限于真实训练与测试环境的缺乏:真实数据因伦理和安全问题难以采集,自主算法训练又需逼真的成像与物理反馈。我们提出ROOM(Realistic Optical Observation in Medicine),一个用于生成逼真支气管镜训练数据的综合仿真框架。通过患者CT扫描,该流程渲染多模态传感器数据,包括带有真实噪声与光反射的RGB图像、度量深度图、表面法向、光流及点云,均在医学相关尺度下完成。我们在两个典型医疗机器人任务中验证了ROOM生成数据的有效性:多视角姿态估计与单目深度估计,展示了现有先进方法在医学场景中需克服的多样化挑战。此外,我们证明ROOM生成的数据可用于微调现有深度估计模型以应对这些挑战,并支持导航等下游应用。预计ROOM将实现跨多样患者解剖结构与操作场景的大规模数据生成,弥补临床难以获取的限制。代码与数据:https://github.com/iamsalvatore/room。
原文摘要 · Abstract (English)
Continuum robots are advancing bronchoscopy procedures by accessing complex lung airways and enabling targeted interventions. However, their development is limited by the lack of realistic training and test environments: Real data is difficult to collect due to ethical constraints and patient safety concerns, and developing autonomy algorithms requires realistic imaging and physical feedback. We present ROOM (Realistic Optical Observation in Medicine), a comprehensive simulation framework designed for generating photorealistic bronchoscopy training data. By leveraging patient CT scans, our pipeline renders multi-modal sensor data including RGB images with realistic noise and light specularities, metric depth maps, surface normals, optical flow and point clouds at medically relevant scales. We validate the data generated by ROOM in two canonical tasks for medical robotics: multi-view pose estimation and monocular depth estimation, demonstrating diverse challenges that state-of-the-art methods must overcome to transfer to these medical settings. Furthermore, we show that the data produced by ROOM can be used to fine-tune existing depth estimation models to overcome these challenges, also enabling other downstream applications such as navigation. We expect that ROOM will enable large-scale data generation across diverse patient anatomies and procedural scenarios that are challenging to capture in clinical settings. Code and data: https://github.com/iamsalvatore/room.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。