arXiv:2606.21600cs.RO2026-06

用离散动作代码引导机器人多任务操作,提升指令跟随与场景适配能力。

VQActFlow: Vector-Quantized Action Mode Steering for Multi-Task Robot Manipulation

论文配图:VQActFlow: Vector-Quantized Action Mode Steering for Multi-Task Robot Manipulation
图 1 · 摘自论文原文
  • 将连续动作分块编码为离散代码,通过变分流匹配生成代码序列。
  • 在推理时通过语言与可行性信号双重引导,准确选择动作模式。
  • 在仿真与真实机器人上均优于连续和离散基线模型。

多任务机器人操作策略从示范中学习极具挑战性,因传统单一网络需根据语言和视觉上下文从异质动作分布中选择不同动作模式,选错模式会导致执行错误任务或不可行动作。将连续动作分块并映射到学习得到的离散代码本,可在表示层面分离不同动作模式,有利于多任务学习。我们提出VQActFlow,一种基于变分流匹配的多任务操作策略,通过向量化动作分块生成代码序列,并在整个生成过程中显式保持对动作模式的偏好。推理时通过引导机制作用于该偏好以调整模式选择:采用无分类器语言条件引导,使策略转向指令指定的动作模式;同时引入学习的代码本评鉴器,提供互补的可行性信号。我们在三个平台评估:LIBERO仿真基准、Unitree G1人形机器人全肢体抓取放置任务,以及ALOHA风格双臂平台执行高接触力任务。在所有基准测试中,VQActFlow均超越连续与离散基线模型。

原文摘要 · Abstract (English)

Multi-task robot manipulation policies are challenging to learn from demonstration because traditionally a single network must select among qualitatively different action modes from a multimodal demonstration distribution, conditioned on language and visual context. A wrong mode selection means executing the wrong task or an action infeasible in the scene. Tokenizing continuous actions into a learned discrete codebook separates these modes at the representation level, offering structural advantages for multi-task learning. We propose VQActFlow, a multi-task manipulation policy that tokenizes action chunks and generates code sequences via Variational Flow Matching. VQActFlow maintains an explicit preference over action modes throughout generation. Inference-time guidance acts on this preference to steer mode commitment. We instantiate this with classifier-free guidance over language conditioning, which steers the policy toward the instructed action mode, and a learned codebook critic that supplies a complementary feasibility signal. We evaluate VQActFlow on three platforms: the LIBERO simulation benchmarks, a Unitree G1 humanoid performing whole-body pick-and-place, and an ALOHA-style bimanual platform performing contact-rich tasks. Across these benchmarks, VQActFlow outperforms both continuous and discrete baselines.

机器人操作动作编码多任务学习生成模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。