开源框架OpenRC实现结肠镜多模态数据同步采集,推动手术自动化研究。
OpenRC: An Open-Source Robotic Colonoscopy Framework for Multimodal Data Acquisition and Autonomy Research
- 改造传统内镜,同步记录视频、操作指令、驱动状态和尖端位置
- 采集1894个远程操作片段,累计约19小时,涵盖10类任务与异常恢复场景
- 支持可复现的多模态机器人结肠镜研究,适合医疗机器人与视觉-语言-动作学习者
结直肠癌筛查依赖结肠镜检查,但现有平台难以系统研究操作者控制、器械运动与视觉反馈之间的耦合动态。这一缺陷限制了机器人结肠镜、医学成像及新兴视觉-语言-动作(VLA)学习范式中可复现的闭环研究。为此,我们提出OpenRC——一个开源模块化机器人结肠镜框架,可对传统内镜进行改造,同时保持临床工作流程。该框架支持视频、操作指令、驱动状态与远端尖端位姿的同步记录。实验验证了运动一致性,并量化了各传感流间的跨模态延迟。基于此平台,我们收集了一个多模态数据集,包含1,894个远程操作回合,总时长约19小时,覆盖10种结构化任务变体:常规导航、失败事件与恢复行为。通过开放硬件设计与对齐的多模态数据集,OpenRC为多模态机器人结肠镜与手术自主性研究提供了可复现的基础。硬件设计与数据集均已发布于https://github.com/artslab2019/openrc-robotic-colonoscopy,数据集已贡献至NVIDIA的Open-H-Embodiment Initiative。
原文摘要 · Abstract (English)
Colorectal cancer screening critically depends on colonoscopy, yet existing platforms offer limited support for systematically studying the coupled dynamics of operator control, instrument motion, and visual feedback. This gap restricts reproducible closed-loop research in robotic colonoscopy, medical imaging, and emerging vision-language-action (VLA) learning paradigms. To address this challenge, we present OpenRC, an open-source modular robotic colonoscopy framework that retrofits conventional scopes while preserving clinical workflow. The framework supports simultaneous recording of video, operator commands, actuation state, and distal tip pose. We experimentally validated motion consistency and quantified cross-modal latency across sensing streams. Using this platform, we collected a multimodal dataset comprising 1,894 teleoperated episodes ~19 hours across 10 structured task variations of routine navigation, failure events, and recovery behaviors. By unifying open hardware and an aligned multimodal dataset, OpenRC provides a reproducible foundation for research in multimodal robotic colonoscopy and surgical autonomy. Both the hardware design and dataset are available at https://github.com/artslab2019/openrc-robotic-colonoscopy. The dataset has been contributed to the Open-H-Embodiment Initiative by NVIDIA.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。