用稀疏专家网络提升视觉语言动作模型性能,兼顾效率与效果
Expertise need not monopolize: Action-Specialized Mixture of Experts for Vision-Language-Action Learning
- 将密集模型转换为稀疏激活的专家混合架构,保留预训练权重
- 在LIBERO和RoboTwin上分别提升1.8%和9.3%,真实场景提升21.5%
- 专家协同工作而非独占,适合需要高效部署的机器人任务
视觉-语言-动作(VLA)模型在机器人操作任务中快速发展并展现出良好能力。然而,扩大模型规模面临两大挑战:(1) 从头训练新VLA模型需大量算力和数据,而当前机器人数据稀缺,因此在扩展过程中充分利用预训练的VLA模型权重至关重要;(2) 实时控制要求在模型容量与计算效率之间取得平衡。为此,我们提出AdaMoE,一种继承自稠密VLA模型预训练权重的专家混合(MoE)架构,通过将前馈层替换为稀疏激活的MoE层来扩展动作专家。AdaMoE采用解耦技术,将专家选择与权重分配分离,借助独立的缩放适配器与传统路由机制协同工作,使专家基于任务相关性被选择,并以可独立控制的权重贡献,实现专家协作而非单个专家主导。实验表明,专业能力无需垄断,通过协作可实现更优性能且保持计算效率。AdaMoE在多个关键基准上持续优于基线模型,在LIBERO上提升1.8%,RoboTwin上提升9.3%,最关键是真实世界实验中实现21.5%的显著提升,验证了其在机器人操作任务中的实际有效性。
原文摘要 · Abstract (English)
Vision-Language-Action (VLA) models are experiencing rapid development and demonstrating promising capabilities in robotic manipulation tasks. However, scaling up VLA models presents several critical challenges: (1) Training new VLA models from scratch demands substantial computational resources and extensive datasets. Given the current scarcity of robot data, it becomes particularly valuable to fully leverage well-pretrained VLA model weights during the scaling process. (2) Real-time control requires carefully balancing model capacity with computational efficiency. To address these challenges, We propose AdaMoE, a Mixture-of-Experts (MoE) architecture that inherits pretrained weights from dense VLA models, and scales up the action expert by substituting the feedforward layers into sparsely activated MoE layers. AdaMoE employs a decoupling technique that decouples expert selection from expert weighting through an independent scale adapter working alongside the traditional router. This enables experts to be selected based on task relevance while contributing with independently controlled weights, allowing collaborative expert utilization rather than winner-takes-all dynamics. Our approach demonstrates that expertise need not monopolize. Instead, through collaborative expert utilization, we can achieve superior performance while maintaining computational efficiency. AdaMoE consistently outperforms the baseline model across key benchmarks, delivering performance gains of 1.8% on LIBERO and 9.3% on RoboTwin. Most importantly, a substantial 21.5% improvement in real-world experiments validates its practical effectiveness for robotic manipulation tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。