通过动态跳过不重要步骤的层,提升机器人操作中视觉语言动作模型的效率。
DySL-VLA: Efficient Vision-Language-Action Model Inference via Dynamic-Static Layer-Skipping for Robot Manipulation
- 按动作重要性动态跳过模型层,关键步骤全算,次要步骤省计算。
- 在Calvin数据集上成功长度提升2.1%,参数减少85.7倍,速度提升3.75倍。
- 适合需要实时推理的机器人应用,尤其对资源受限场景友好。
视觉-语言-动作(VLA)模型通过融合语言模型的推理与视觉模型的3D理解,在机器人操作任务中表现优异。然而,其高计算开销限制了实际部署中的实时性能。我们观察到任务中不同动作的重要性各异:关键步骤需高精度,非关键步骤可容忍更大偏差。基于此,提出DySL-VLA框架,通过动态跳过各动作对应的模型层来降低计算成本。该框架将层分为信息型层(始终执行)和增量型层(可选择跳过)。为智能判断何时跳层而不损失精度,设计先验-后验跳层引导机制。同时提出一种两阶段感知跳层的知识蒸馏算法,高效训练标准VLA转化为DySL-VLA。实验显示,DySL-VLA在Calvin数据集上相较Deer-VLA成功率提升2.1%,可训练参数减少85.7倍,且在相同准确率下比RoboFlamingo快3.75倍。代码已开源。
原文摘要 · Abstract (English)
Vision-Language-Action (VLA) models have shown remarkable success in robotic tasks like manipulation by fusing a language model's reasoning with a vision model's 3D understanding. However, their high computational cost remains a major obstacle for real-world applications that require real-time performance. We observe that the actions within a task have varying levels of importance: critical steps demand high precision, while less important ones can tolerate more variance. Leveraging this insight, we propose DySL-VLA, a novel framework that addresses computational cost by dynamically skipping VLA layers based on each action's importance. DySL-VLA categorizes its layers into two types: informative layers, which are consistently executed, and incremental layers, which can be selectively skipped. To intelligently skip layers without sacrificing accuracy, we invent a prior-post skipping guidance mechanism to determine when to initiate layer-skipping. We also propose a skip-aware two-stage knowledge distillation algorithm to efficiently train a standard VLA into a DySL-VLA. Our experiments indicate that DySL-VLA achieves 2.1% improvement in success length over Deer-VLA on the Calvin dataset, while simultaneously reducing trainable parameters by a factor of 85.7 and providing a 3.75x speedup relative to the RoboFlamingo baseline at iso-accuracy. Our code is available on https://github.com/PKU-SEC-Lab/DYSL_VLA.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。