从直播视频学网球轮椅导航，97.67%成功率
Learning Wheelchair Tennis Navigation from Broadcast Videos with Domain Knowledge Transfer and Diffusion Motion Planning
- 用多视角视频重建3D任务空间，转为2D规划闭环
- 在真实场地实测中达到68.49%实时导航成功率
- 适合轮椅辅助、运动机器人等具身智能场景
本文提出一种新颖且可泛化的零样本知识迁移框架，通过对抗约束和分布外图像轨迹，将网络视频中的专家体育导航策略提炼至机器人系统。该流程通过多视角重建完整3D任务空间，将其映射到2D图像空间，在此空间内闭合规划回路，并将受约束的运动轨迹回传至任务空间。此外，我们证明学习到的策略可作为局部规划器与位置控制结合使用。该框架应用于轮椅网球导航问题，引导轮椅进入击球区域。在物理机器人上，对真实记录的网球轨迹实现97.67%的导航成功率；在全尺寸网球场的真实实时实验中，成功率达68.49%。
原文摘要 · Abstract (English)
In this paper, we propose a novel and generalizable zero-shot knowledge transfer framework that distills expert sports navigation strategies from web videos into robotic systems with adversarial constraints and out-of-distribution image trajectories. Our pipeline enables diffusion-based imitation learning by reconstructing the full 3D task space from multiple partial views, warping it into 2D image space, closing the planning loop within this 2D space, and transfer constrained motion of interest back to task space. Additionally, we demonstrate that the learned policy can serve as a local planner in conjunction with position control. We apply this framework in the wheelchair tennis navigation problem to guide the wheelchair into the ball-hitting region. Our pipeline achieves a navigation success rate of 97.67% in reaching real-world recorded tennis ball trajectories with a physical robot wheelchair, and achieve a success rate of 68.49% in a real-world, real-time experiment on a full-sized tennis court.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。