arXiv:2503.20384cs.ROcs.AI2025-03AAAI被引 79

通过动态选择神经层,让机器人模型更省力地完成任务。

MoLe-VLA: Dynamic Layer-skipping Vision Language Action Model via Mixture-of-Layers for Efficient Robot Manipulation

  • 用空间时间感知路由机制,只激活必要神经层。
  • 比普通模型快5.6倍,任务成功率平均提升8%。
  • 适合需要高效推理的机器人实操场景。

多模态大语言模型(MLLM)在理解复杂语言与视觉信息方面表现优异,使通用型机器人系统能够解析指令并执行具身任务。然而,其实际部署受限于巨大的计算与存储开销。近期关于大语言模型各层同质性模式的研究启发了稀疏化技术,如提前退出和令牌剪枝。但这些方法常忽略最终层中编码的对下游机器人任务至关重要的语义信息。受神经科学中‘浅脑假说’(SBH)及专家混合模型稀疏化思想的启发,我们将每层视为一个专家,提出一种动态层激活的视觉-语言-动作模型(MoLe-VLA)。设计了时空感知路由(STAR)机制,根据机器人当前状态选择性激活部分层,模拟大脑中专门负责认知与因果推理的信号通路。为弥补模型因剪枝损失的认知能力,引入认知自知识蒸馏(CogKD)框架,利用认知特征增强对任务需求的理解,提升任务相关动作序列生成能力。在RLBench仿真与真实环境中的大量实验表明,MoLe-VLA在效率与性能上均显著优于标准大模型:在10个任务上平均成功率提升8%,计算成本降低最高达5.6倍。

原文摘要 · Abstract (English)

Multimodal Large Language Models (MLLMs) excel in understanding complex language and visual data, enabling generalist robotic systems to interpret instructions and perform embodied tasks. Nevertheless, their real-world deployment is hindered by substantial computational and storage demands. Recent insights into the homogeneous patterns in the LLM layer have inspired sparsification techniques to address these challenges, such as early exit and token pruning. However, these methods often neglect the critical role of the final layers that encode the semantic information most relevant to downstream robotic tasks. Aligning with the recent breakthrough of the Shallow Brain Hypothesis (SBH) in neuroscience and the mixture of experts in model sparsification, we conceptualize each LLM layer as an expert and propose a Mixture-of-Layers Vision-Language-Action model (MoLe-VLA, or simply MoLe) architecture for dynamic LLM layer activation. We introduce a Spatial-Temporal Aware Router (STAR) for MoLe to selectively activate only parts of the layers based on the robot's current state, mimicking the brain's distinct signal pathways specialized for cognition and causal reasoning. Additionally, to compensate for the cognitive ability of LLMs lost in MoLe, we devise a Cognition Self-Knowledge Distillation (CogKD) framework. CogKD enhances the understanding of task demands and improves the generation of task-relevant action sequences by leveraging cognitive features. Extensive experiments conducted in both RLBench simulation and real-world environments demonstrate the superiority of MoLe-VLA in both efficiency and performance. Specifically, MoLe-VLA achieves an 8% improvement in the mean success rate across ten tasks while reducing computational costs by up to x5.6 compared to standard LLMs.

机器人稀疏模型大模型压缩

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。