arXiv:2603.07647cs.RO2026-03被引 5

给冻结的视觉语言动作模型加动态记忆,让长时序操作更稳定。

TempoFit: Plug-and-Play Layer-Wise Temporal KV Memory for Long-Horizon Vision-Language-Action Manipulation

  • 复用模型自身注意力键值对作分层时间记忆,无需新增参数或训练。
  • 在LIBERO-LONG上提升平均成功率最高达4.0%,延迟几乎不变。
  • 适合需要长期推理但无法重训练的机器人控制场景。

预训练的视觉-语言-动作(VLA)策略在单步操作中表现强劲,但在存在遮挡、状态混淆和细微动作后变化的非马尔可夫长时序场景中,其无记忆推理方式易出错。现有方法或堆叠帧导致视觉令牌和延迟增长,或引入需重新训练的时间接口模块。我们提出TempoFit,一种无需训练的时间重构方案,通过状态级记忆升级冻结的VLA。核心思想是:前缀注意力的键值对本身已是模型原生、内容可寻址的运行时状态;跨时间步复用这些键值对即可引入历史信息,而无需新增令牌或可训练模块。TempoFit在选定中间层存储分层的先进先出前缀键值对,采用固定的时间间隙感知偏置(FGTB),模仿自然语言处理中的位置偏置,保持决策以当前时刻为主导,并通过预注意力残差加载与归一化保持缩放,避免冻结权重下的分布偏移。在LIBERO-LONG上,TempoFit使强预训练骨干网络的平均成功率最高提升4.0%,同时保持近实时延迟,并能一致迁移至CALVIN及真实机器人长时序任务。

原文摘要 · Abstract (English)

Pretrained Vision-Language-Action (VLA) policies have achieved strong single-step manipulation, but their inference remains largely memoryless, which is brittle in non-Markovian long-horizon settings with occlusion, state aliasing, and subtle post-action changes. Prior approaches inject history either by stacking frames, which scales visual tokens and latency while adding near-duplicate pixels, or by learning additional temporal interfaces that require (re-)training and may break the original single-frame inference graph. We present TempoFit, a training-free temporal retrofit that upgrades frozen VLAs through state-level memory. Our key insight is that prefix attention K/V already form a model-native, content-addressable runtime state; reusing them across timesteps introduces history without new tokens or trainable modules. TempoFit stores layer-wise FIFO prefix K/V at selected intermediate layers, performs parameter-free K-to-K retrieval with Frame-Gap Temporal Bias (FGTB), a fixed recency bias inspired by positional biases in NLP, to keep decisions present-dominant, and injects the retrieved context via pre-attention residual loading with norm-preserving rescaling to avoid distribution shift under frozen weights. On LIBERO-LONG, TempoFit improves strong pretrained backbones by up to +4.0% average success rate while maintaining near-real-time latency, and it transfers consistently to CALVIN and real-robot long-horizon tasks.

长时序控制记忆机制视觉语言动作零样本适配

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。