让机器人实时执行更顺滑,通过预测未来视觉与状态来对齐动作
FutureRTC: Real-Time Robot Execution with Anticipatory-Conditioned Action Chunking

- 用运动作为物理先验,预测执行时的视觉和本体感觉状态
- 在延迟下任务成功率提升,轨迹更平滑,速度更快
- 无需修改原策略,适合部署于真实机器人系统
视觉-语言-动作(VLA)策略的实时部署需异步执行,即后续动作块在当前块执行时并行计算,导致预测与执行不同步,表现为块间不连续。现有方法或仅表面平滑边界,或需昂贵策略优化,或仅前向预测本体感觉状态而忽略关键视觉观察。本文提出 extbf{FutureRTC},一种即插即用的适配框架,在不修改底层策略的前提下,预测异步控制下的执行时刻观测与状态。具体包括:状态修正模块补偿滚动预测与实际执行时本体感觉状态的差异;观察预测模块利用机器人运动作为显式物理先验,通过运动感知特征传输与重建,预测执行时刻的视觉表征。此外,引入策略一致性损失,使基于预测上下文生成的动作块与预期执行时刻输入下VLA策略产生的动作块对齐。大量模拟与真实环境实验表明,FutureRTC显著提升对推理延迟的鲁棒性,实现更平滑轨迹、更快执行及更高任务成功率。
原文摘要 · Abstract (English)
Real-time deployment of Vision-Language-Action (VLA) policies necessitates asynchronous execution, wherein subsequent action chunks are computed concurrently with the execution of the current chunk, leading to prediction-execution misalignment and manifesting as inter-chunk discontinuities. Existing methods either superficially smooth chunk boundaries, require costly policy optimization, or exclusively forward-predict proprioceptive states yet neglect critical visual observations. In this paper, we propose \textbf{FutureRTC}, a plug-and-play adaptation framework that predicts execution-time observations and states for asynchronous VLA control without modifying the underlying policy. Specifically, FutureRTC features a state correction module to compensate for the discrepancy between rolled-forward and actual execution-time proprioceptive states and an observation prediction module that forecasts execution-time visual representations by leveraging robot motion as an explicit physical prior through motion-aware feature transport and reconstruction. Furthermore, we introduce a policy consistency loss to align the action chunks generated from predicted contexts with those produced under the expected execution-time inputs of the VLA policy. Extensive experiments across simulated and real-world environments demonstrate that FutureRTC achieves superior robustness to inference delays, resulting in smoother trajectories, faster execution, and consistently higher task success rates.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。