把任务知识全塞进条件路径,让动作主干变通用,性能不降反升。
Decoupled Action Expert: Confining Task Knowledge to the Conditioning Pathway
- 将任务知识限制在条件路径,冻结主干网络复用所有任务。
- 500万参数小模型表现媲美24400万参数大模型,性能相当。
- 适合追求轻量化、高复用性的机器人控制研究者。
当前视觉-语言-动作模型多采用包含数亿参数的扩散或流匹配主干进行动作生成。然而,与图像合成中数百万像素的输出不同,操作策略仅生成低维、物理相关的短序列动作值,目标远为简单,不应需要如此庞大容量。我们验证了这一直觉,表明任务特定知识可完全保留在条件路径中,使动作主干保持任务无关。为此,我们提出解耦训练方案:先在无观测前向运动学数据上预训练通用动作头,再冻结主干,仅训练下游任务的条件路径。以扩散策略(Diffusion Policy)为测试平台,在MimicGen和LIBERO数据集上,共享的单一冻结主干性能可媲美常规训练模型。消融实验表明,预训练信号(关节位置、末端执行器位姿或无条件)对下游性能无影响,说明主干仅学习通用轨迹结构。进一步地,我们将原24400万参数的U-Net替换为500万参数的MLP主干,性能仍持平或超越原模型,质疑了当前视觉-语言-动作设计中对动作生成的大规模容量投入。
原文摘要 · Abstract (English)
Many recent Vision-Language-Action models employ diffusion or flow-matching backbones with hundreds of millions of parameters for action generation. However, unlike image synthesis where the output spans millions of diverse pixels, a manipulation policy generates only short sequences of low-dimensional, physically correlated action values, a far simpler target that should not demand such capacity. We confirm this intuition and show that task-specific knowledge in these policies can be fully confined to the conditioning pathway, leaving the action backbone task-agnostic. To establish this, we introduce a decoupled training recipe: a general-purpose action head is first pretrained on observation-free forward-kinematics data, then frozen while only the conditioning pathway is trained for downstream tasks. Using Diffusion Policy as a testbed, we show that on both MimicGen and LIBERO, a single frozen backbone shared across all tasks matches normally trained counterparts. This confirms that the action expert encodes little task-specific knowledge. Ablations show that the specific pretraining signal (joint positions, end-effector poses, or no conditioning at all) has no effect on downstream performance, indicating that the backbone learns only general trajectory structure. Pushing this finding further, we replace the 244M U-Net in Diffusion Policy with a 5M-parameter MLP backbone that matches or exceeds its performance, calling into question the large capacity budgets allocated to action generation in current VLA designs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。