arXiv:2607.15275cs.ROcs.AI2026-07被引 7

让机器人模型记忆长达8000步的视觉动作上下文,性能提升87%。

RoboTTT: Context Scaling for Robot Policies

论文配图:RoboTTT: Context Scaling for Robot Policies
图 1 · 摘自论文原文
  • 用测试时训练将上下文扩展到8000步,不增加推理延迟
  • 在真实机器人上完成5分钟10阶段装配任务,基线无法达成
  • 支持一次示范模仿、实时策略优化,适合长程复杂任务

近期机器人基础模型仅使用单步或短历史视觉动作上下文。我们提出测试时训练机器人策略(RoboTTT),一种可将视觉动作上下文扩展至8000个时间步的模型与训练方法,比现有最优方案长三个数量级,且不增加推理延迟。在此上下文长度下,实现了首次示范的上下文内模仿、实时策略改进、抗扰动鲁棒性以及多阶段长时程任务更强表现。首次观察到预训练上下文长度增加时闭环性能持续提升。核心是将测试时训练融入视觉-语言-动作模型,其循环状态由快速权重构成,通过梯度下降在训练和推理时更新,将历史压缩至参数空间并实现长上下文条件化。训练方面结合序列动作强迫与截断反向传播通过时间。在挑战性真实机器人操作任务中,相比单步上下文基线,性能提升87%,并首次完整完成耗时五分钟、包含十阶段的装配任务。使用8000步预训练的RoboTTT比1000步预训练版本提升62%,表明上下文长度是机器人基础模型的新缩放轴。

原文摘要 · Abstract (English)

Recent robot foundation models operate with single-step or short-history visuomotor context. We introduce Test-Time-Training Robot Policies (RoboTTT), a robot model and training recipe that scale visuomotor context to 8K timesteps, three orders of magnitude beyond state-of-the-art policies, without growing inference latency. At this context length, we unlock new robot capabilities: one-shot in-context imitation from human video demonstrations, on-the-fly policy improvement, robustness to perturbations, and stronger performance on multi-stage, long-horizon tasks. We also observe, for the first time, steady gains in closed-loop performance as pretraining context length scales. At its core, RoboTTT integrates Test-Time Training into robot foundation models such as Vision-Language-Action policies, yielding a sequence model whose recurrent state consists of fast weights, parameters updated by gradient descent during both training and inference, compressing histories into weight space and retrieving contextual information for long-context conditioning. To scale training context length, the recipe combines sequence action forcing with truncated backpropagation through time. On challenging real-robot manipulation tasks, RoboTTT improves overall performance by 87% over the single-step context baseline and fully completes a five-minute, ten-stage assembly task, which no baseline ever does. RoboTTT trained with 8K-timestep context outperforms the same model pretrained with 1K timesteps by 62%, suggesting context length as a new scaling axis for robot foundation models. Videos are available at https://research.nvidia.com/labs/gear/robottt/

机器人长上下文测试时训练强化学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。