让四足机器人在0.8-1.2秒内精准拦截动态球体,突破传统速度追踪的时序限制。
Spatiotemporal Agility: Time-Constrained Reinforcement Learning for Vision-Guided Dynamic Quadrupedal Interception

- 直接以目标位置和时间作为强化学习输入,跳过中间速度指令
- 在2米内落点、0.8~1.2秒飞行时间条件下成功率显著提升
- 端到端闭环系统融合多视角感知与低延迟控制,适合真实动态任务
腿式机器人需具备强健的时空敏捷性,在有限时间内感知并交互复杂动态环境。然而,现有四足运动方法多依赖速度追踪策略,难以在严格时间约束下精准到达目标。此外,实时感知与敏捷运动融合面临传感器延迟与处理延迟挑战。为系统研究此类敏捷性,我们提出一个面向腿式机器人的高动态球体拦截任务。本文设计了一种集成框架,结合视觉模块预测落点与时间,并采用直接以位置和时间条件化的强化学习运动策略,而非中间速度指令。该工作实现了一个完整的实时机器人拦截系统,整合多相机感知、在线轨迹预测、低延迟目标通信与仿真到现实的运动控制,形成闭环部署流程。通过显式预测未来时空目标,缓解了动态拦截中的感知延迟。我们在四足机器人上开展大量球体拦截实验,对比速度追踪基线,在落点距2米内、飞行时间0.8至1.2秒的条件下,本方法成功率更高,表明机器人成功完成动态球体拦截任务。此外,部署后性能差距更小,说明仿真到现实的泛化能力更强。
原文摘要 · Abstract (English)
Legged robots require robust agility to perceive and interact with complex and dynamic environments within a constrained time. However, most existing quadruped locomotion works rely on velocity-tracking policy, which struggle to reach precise targets within strict temporal constraints. Moreover, integrating real-time perception with agile locomotion for highly dynamic targets remains challenging due to sensor latency and processing delays. To concretely study and benchmark such agility in dynamic settings, we introduce a challenging ball-catching task for legged robots. This paper proposes an integrated framework that combines a vision module for landing point and time prediction with a direct position and time conditioned RL locomotion policy, instead of intermediate velocity commands. Beyond the method design, this work presents a system-level contribution that completes real-time robotic interception system that integrates multi-camera perception, online trajectory prediction, low-latency target communication, and sim-to-real locomotion control into a closed-loop deployment pipeline. By explicitly predicting the future spatial-temporal target, our approach mitigates perception latency during dynamic interception. We conducted extensive ball-catching experiments for the legged robot. Through comparative experiments against a velocity-tracking baseline, our direct target-conditioned approach achieves a higher success rate in catching balls with predicted landing spots within 2 meters and flight times between 0.8 and 1.2 seconds. This shows that the robot has successfully completed the dynamic ball-catching task under our tested setup. Furthermore, our policy exhibits a smaller performance gap after deployment, suggesting improved sim-to-real behavior in these trials.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。