让视觉语言动作模型学会多技能，解决合并专家时性能暴跌问题。
MergeVLA: Cross-Skill Model Merging Toward a Generalist Vision-Language-Action Agent
- 用任务掩码控制稀疏激活的LoRA适配器，保持参数一致性。
- 用交叉注意力替代自注意力，使动作专家局部可组合，避免信息扩散。
- 测试时通过观察自动选择任务头，实现无监督任务识别。
近期视觉-语言-动作（VLA）模型通过数百万机器人演示微调视觉语言模型，虽在单任务或单一机器人上表现良好，但扩展至多技能场景仍困难：直接合并不同任务训练的专家会导致成功率接近零。我们通过分析微调中可学习参数的演变，发现两个关键阻碍因素：(1) 微调使视觉语言模型骨干中的LoRA适配器朝任务特异性方向发散，超出现有合并方法的统一能力；(2) 动作专家通过自注意力反馈建立层间依赖，导致任务信息跨层传播，无法模块化重组。为此，我们提出MergeVLA，一种从设计上保障可合并性的VLA架构。其采用任务掩码控制的稀疏激活LoRA适配器，维持参数一致性并减少不可调和冲突；动作专家改用仅含交叉注意力的块，使专长局部化、可组合。当任务未知时，通过测试时任务路由器根据初始观测自适应选择合适的任务掩码与专家头，实现无监督任务推断。在真实SO101机械臂上的LIBERO、LIBERO-Plus、RoboTwin及多任务实验中，MergeVLA性能达到甚至超过单独微调的专家，展现出对任务、体感和环境的鲁棒泛化能力。
原文摘要 · Abstract (English)
Recent Vision-Language-Action (VLA) models reformulate vision-language models by tuning them with millions of robotic demonstrations. While they perform well when fine-tuned for a single embodiment or task family, extending them to multi-skill settings remains challenging: directly merging VLA experts trained on different tasks results in near-zero success rates. This raises a fundamental question: what prevents VLAs from mastering multiple skills within one model? With an empirical decomposition of learnable parameters during VLA fine-tuning, we identify two key sources of non-mergeability: (1) Finetuning drives LoRA adapters in the VLM backbone toward divergent, task-specific directions beyond the capacity of existing merging methods to unify. (2) Action experts develop inter-block dependencies through self-attention feedback, causing task information to spread across layers and preventing modular recombination. To address these challenges, we present MergeVLA, a merging-oriented VLA architecture that preserves mergeability by design. MergeVLA introduces sparsely activated LoRA adapters via task masks to retain consistent parameters and reduce irreconcilable conflicts in the VLM. Its action expert replaces self-attention with cross-attention-only blocks to keep specialization localized and composable. When the task is unknown, it uses a test-time task router to adaptively select the appropriate task mask and expert head from the initial observation, enabling unsupervised task inference. Across LIBERO, LIBERO-Plus, RoboTwin, and multi-task experiments on the real SO101 robotic arm, MergeVLA achieves performance comparable to or even exceeding individually finetuned experts, demonstrating robust generalization across tasks, embodiments, and environments. Project page: https://mergevla.github.io/
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。