无需训练数据,动态加速视觉语言动作模型推理
AdaVLA: Adaptive Step Flow Matching for Training-free Acceleration of Vision-Language-Action Models

- 基于轨迹曲率设计新指标,实时评估动作生成置信度
- 在Jetson设备上实现1.87~2.24倍加速,成功率几乎不变
- 适合隐私受限、无法获取训练数据的机器人部署场景
视觉语言动作(VLA)模型依托视觉语言模型(VLM),通过海量网络知识与多模态推理显著提升了机器人能力。然而,其高计算开销限制了设备端部署,难以实现实时响应。现有加速方法多依赖微调或训练数据,但常因隐私与产权问题不可用。尽管基于流匹配的VLA已成为扩散模型的有效替代,当前加速仍集中于VLM推理,未解决流匹配推理中固有的迭代常微分方程(ODE)求解问题。为此,我们提出AdaVLA,一种在线、免训练的自适应框架,用于快速且准确的流匹配式VLA模型加速。我们引入一种源自流匹配轨迹曲率的新指标,量化推理过程中的动作生成置信度。该指标使推理步数动态减少,并通过高效计算的重要性评估自适应调整MLP剪枝比例,全程无需训练数据。在Jetson AGX Orin设备上的LIBERO基准测试显示,本方法对π₀.₅和X-VLA分别实现1.87×和2.24×加速,成功率几乎无损。此外,我们在真实机器人任务中验证了SmolVLA的鲁棒性。
原文摘要 · Abstract (English)
Vision-Language-Action (VLA) models, built upon Vision-Language Models (VLMs), have significantly enhanced robotic capabilities by leveraging internet-scale knowledge and multimodal reasoning. However, the intensive computational overhead of VLAs constrains on-device deployment, hindering real-time responses to environmental changes. While various acceleration techniques have been proposed, they often rely on fine-tuning or access to training datasets, which are frequently unavailable due to privacy and proprietary concerns. Moreover, although flow-matching-based VLAs have emerged as efficient alternatives to standard diffusion models, current acceleration efforts largely target VLM inference costs, failing to address the iterative ODE solving process inherent in flow matching inference. To address these limitations, we propose AdaVLA, an online, training-free adaptive framework for fast yet accurate flow-matching-based Vision-Language-Action models. We introduce a novel metric derived from the flow matching trajectory curvature to quantify action generation confidence during inference. This metric enables the dynamic reduction of inference steps and the adaptive adjustment of MLP pruning ratios through an efficiently computed importance evaluation, requiring no access to training data. Experimental results on the LIBERO benchmark using a Jetson AGX Orin device demonstrate that our method achieves $1.87\times$ and $2.24\times$ speedups for $π_{0.5}$ and X-VLA, respectively, with negligible degradation in success rates. Furthermore, we validate the robustness of our approach on real-world robotic tasks using SmolVLA.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。