用Mamba替代Transformer,让机器人视觉语言动作模型更准且更快。
Mamba-based Selective State Space Modeling Improves the Accuracy-Complexity Tradeoff of SmolVLA Vision-Language-Action Experts

- 用Mamba的选通状态空间机制替代Transformer注意力,提升效率。
- 长执行周期下(N=50)成功率领先基准7.8%,实现实时部署。
- 参数量减少24%,适合资源受限的机器人实时控制场景。
视觉-语言-动作(VLA)模型在任务成功率与策略调用频率间存在关键权衡。单次推理执行一个动作(N=1)虽精准但计算开销大,难以实现实时;而长时间执行多个动作后再重规划(N≫1)虽降低复杂度,却损害成功率。本文将Mamba的选通状态空间建模引入广受认可的SmolVLA模型的动作专家中,替代其因果自注意力机制。我们在LIBERO基准上对Mamba和Transformer专家在三种执行周期N∈{1,25,50}下进行评估,分别对应高、中、低计算复杂度。结果表明,随着执行周期增长,Mamba的优势愈发显著:当每50步重规划一次(即可行实时部署)时,Mamba专家比基线高出7.8%;在25步时领先3.7%;而在每步重规划(N=1)下,其成功率与基准相当,同时整体模型参数量减少24%。
原文摘要 · Abstract (English)
Vision-language-action (VLA) models face a crucial tradeoff between their task success rate and the policy-call frequency. Executing a single action per inference ($N=1$) enables accurate robot control but comes at the cost of huge compute time overheads, making real-time implementation infeasible. On the other hand, executing longer action horizons before replanning ($N\gg1$) reduces compute complexity, but inevitably degrades the system's success rate. In order to improve the VLA accuracy-complexity tradeoff, this paper investigates Mamba's selective state-space modeling as an alternative to causal self-attention within the action expert of the popular SmolVLA model, widely used as a reference model for its highly accurate yet low complexity nature. We evaluate both the Mamba- and Transformer-based experts on the widely-adopted LIBERO benchmark suites across three execution horizons $N\!\in\!\{1,25,50\}$, respectively corresponding to high, moderate and low compute complexities. Our results remarkably show that the advantage of the Mamba expert increases with the execution horizon, indicating significant success retention under long execution horizons $N = 50$ and $N = 25$. When $N = 50$ actions are executed before replanning (i.e., corresponding to feasible real-time deployment), the Mamba expert outperforms the Transformer baseline by $7.8\%$. In addition, when $N = 25$ actions are executed before replanning, our Mamba expert outperforms the Transformer baseline by $3.7\%$. Finally, under per-action replanning ($N=1$), our Mamba variant matches the Transformer-based mean success rate while significantly reducing the overall model parameter complexity by $24\%$ thanks to Mamba's compute-efficient nature.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。