arXiv:2607.00808cs.LG2026-07

拆解视频中局部运动,提升强化学习预训练的泛化能力

Local Motion Matters: A Deconstruct-Recompose Paradigm for Reinforcement Learning Pre-training from Videos

论文配图:Local Motion Matters: A Deconstruct-Recompose Paradigm for Reinforcement Learning Pre-training from Videos
图 1 · 摘自论文原文
  • 将动作分解为可追踪的局部原子单元,用双注意力编码器学习其时空特征
  • 通过可学习的运动聚合令牌重组局部动作,实现跨任务高效迁移
  • 在机器人控制与操作任务中显著提升样本效率,适合多场景强化学习研究

在大规模视频上进行预训练以提升强化学习效率具有潜力但依然面临挑战。现有方法通常将智能体视为不可分割的整体,全局建模运动模式,这种建模方式与形态紧密耦合,阻碍了跨域迁移。相反,尽管全局运动差异显著,不同智能体的局部组件却表现出相似的运动模式。基于此洞察,我们提出一种新的拆解-重构范式(Deconstruct-Recompose Paradigm, DRP),用于学习可迁移的局部运动表征。具体而言,在拆解阶段,我们识别多个局部点并追踪其逐帧运动,将每个定义为原子动作(Atomic Action)。引入双注意力编码器(DAE)从这些原子动作中学习局部运动表征,捕捉其时空关系。在重构阶段,通过隐动态模型学习,利用可学习的运动聚合令牌(MAT)组合局部运动表征。此外,一个适配器将局部运动与下游动作特定动态相连接,加速策略学习。大量实验表明,该方法能有效迁移到多种机器人控制与操作任务中,显著提升样本效率和性能。

原文摘要 · Abstract (English)

Pre-training on large-scale videos to improve reinforcement learning efficiency is promising yet remains challenging. Existing methods typically treat the agent as an indivisible entity, modeling motion patterns globally. Such global modeling is tightly coupled with the morphology, hindering transfer across domains. In contrast, despite the vast disparity in global motions, the local components exhibit similar motion patterns across different agents. Building on this insight, we propose a novel Deconstruct-Recompose Paradigm (DRP) for learning transferable local motion representations. Specifically, in the Deconstruct phase, we identify multiple local points and track their frame-wise motions, defining each as an Atomic Action. We introduce a Dual-Attention Encoder (DAE) to learn local motion representations from these Atomic Actions, capturing their spatiotemporal relationships. In the Recompose phase, we compose local motion representations with a learnable Motion Aggregation Token [MAT] via latent dynamics model learning. Additionally, an adapter bridges local motion and downstream action-specific dynamics to accelerate policy learning. Extensive experiments demonstrate that our method effectively transfers to diverse robotic control and manipulation tasks, significantly improving sample efficiency and performance.

强化学习视频预训练运动表征迁移学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。