用AR手部追踪和仿真平台大规模采集外科手术机器人数据,提升效率并降低存储成本。
dARt Vinci: Egocentric Data Collection for Surgical Robot Learning at Scale
- 通过AR手追踪与高保真物理引擎实现无实物机器人的沉浸式数据采集
- 数据生成速度比真实机器人快41%,实验时间减少10%,存储量缩小400倍
- 适合需要海量高质量手术数据的研究者,尤其适用于远程操控与强化学习训练
数据稀缺长期制约机器人学习发展,尤其在安全敏感的外科领域,高质量数据获取困难。本文提出dARt Vinci,一个面向外科机器人学习的可扩展数据采集平台。系统结合增强现实(AR)手部追踪与高保真物理引擎,无需实体机器人即可捕捉基础手术动作中的细微操作;同时利用AR的体感追踪与内容叠加能力,实现更自然的自视角数据采集。用户研究表明,相比真实机器人环境,所有任务的数据吞吐量平均提升41%,总实验时间平均缩短10%,任务负荷的时间压力显著改善,结果具有统计显著性。所收集数据体积缩小超400倍,存储需求大幅降低,同时采样频率翻倍,具备更高效率与实用性。
原文摘要 · Abstract (English)
Data scarcity has long been an issue in the robot learning community. Particularly, in safety-critical domains like surgical applications, obtaining high-quality data can be especially difficult. It poses challenges to researchers seeking to exploit recent advancements in reinforcement learning and imitation learning, which have greatly improved generalizability and enabled robots to conduct tasks autonomously. We introduce dARt Vinci, a scalable data collection platform for robot learning in surgical settings. The system uses Augmented Reality (AR) hand tracking and a high-fidelity physics engine to capture subtle maneuvers in primitive surgical tasks: By eliminating the need for a physical robot setup and providing flexibility in terms of time, space, and hardware resources-such as multiview sensors and actuators-specialized simulation is a viable alternative. At the same time, AR allows the robot data collection to be more egocentric, supported by its body tracking and content overlaying capabilities. Our user study confirms the proposed system's efficiency and usability, where we use widely-used primitive tasks for training teleoperation with da Vinci surgical robots. Data throughput improves across all tasks compared to real robot settings by 41% on average. The total experiment time is reduced by an average of 10%. The temporal demand in the task load survey is improved. These gains are statistically significant. Additionally, the collected data is over 400 times smaller in size, requiring far less storage while achieving double the frequency.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。