让机器人视觉语言动作模型按需选择最优特征层,提升控制精度。
LayerRoute: Action-Conditioned Mixture-of-Layers Routing for Vision-Language-Action Policies

- 根据动作需求动态组合视觉语言模型的多层特征
- 在真实世界任务中最高提升7.2分,参数增加不足0.4%
- 适合需要精细感知与决策的机器人控制研究者
视觉语言动作(VLA)策略利用预训练视觉语言模型(VLM)指导机器人动作生成。VLM 提供从局部视觉几何到抽象语义的层次化表示,不同操作任务可能需要不同的层表示组合。同时,动作模块在计算过程中保留中间状态,可为后续决策提供信息。然而现有接口对表示访问支持有限:VLM 特征仅通过固定层分配给各动作层,而中间动作状态仅隐式通过残差流传递,无法显式复用。本文提出 LayerRoute,一种动作条件的表征路由接口,实现对 VLM 层和动作表示的自适应访问。层混合路由器动态构建缓存的 VLM 表示混合,动作状态重读机制复用早期动作表示。在多种仿真与真实世界基准上,LayerRoute 稳定提升 StarVLA-π 与 π₀.₅,最大在 LIBERO Long 上提升 7.2 分,仅增加 0.31% / 3.87% 参数量。消融实验验证动作条件路由的有效性,路由分析揭示了动作层与任务设置下的结构化分配模式。
原文摘要 · Abstract (English)
Vision-Language-Action (VLA) policies leverage pretrained vision-language models (VLMs) to guide action generation for robot control. VLMs provide hierarchical visual-semantic representations that evolve across layers, from local visual geometry to abstract, language-aligned semantics; different manipulation tasks may therefore require different mixtures of layer representations. Meanwhile, the action module maintains intermediate representations that evolve throughout action computation and may provide useful information for subsequent decisions. However, existing VLA interfaces offer limited flexibility in representation access: VLM information is exposed through fixed layer assignments for each action layer, while intermediate action states are only propagated implicitly through residual streams without explicit reuse. We introduce LayerRoute, an action-conditioned representation routing interface that enables adaptive access to VLM layers and action representations. The Layer Mixture Router dynamically forms mixtures of cached VLM representations, while Action-State Reread reuses earlier action representations. Across diverse simulation and real-world benchmarks, LayerRoute consistently improves StarVLA-$\pi$ and $\pi_{0.5}$, achieving up to 7.2 gains on LIBERO Long with only 0.31% / 3.87% additional parameters. Ablation studies validate the benefit of action-conditioned layer routing, while routing analyses reveal structured allocation patterns across action layers and task settings.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。