让视觉语言动作模型学会适时计算,提升机器人控制效率
ElegantVLA: Learning When to Think for Efficient Vision-Language-Action Models

- 根据视觉语言稳定性动态调整计算强度,分五级调度
- 动作生成阶段复用中间状态,关键阶段保留完整优化
- 无需重训练,实测提速最高3.77倍,控制频率翻倍
视觉-语言-动作(VLA)模型是通用机器人控制的强大范式,但其高计算开销和低控制频率限制了实时操作,尤其在大尺寸视觉语言主干网络与迭代动作头每步均运行时。现有加速方法多针对单一组件优化或依赖固定规则,忽略序列体化控制中不同步骤的非均匀推理需求。受人类运动控制启发,我们提出ElegantVLA:一种插件式相位自适应推理框架,通过模型内动态计算调度实现加速。该框架引入轻量级调度器,综合观察时序表征相似性、机器人运动线索与任务进度,协调分配视觉编码器、大语言模型与动作头的计算资源。感知-语言推理采用五级计算模式,从完全重算到多步时序复用;动作生成则采用三级去噪模式,在稳定运动中复用中间去噪状态,关键阶段保留完整精炼。该框架无需修改或重新训练基础模型,在GR00T与CogACT上分别实现最高2.55倍和3.77倍加速,并在六个真实世界GR00T任务中将计算量减少2.18倍,控制频率从13.8 Hz提升至26.3 Hz。
原文摘要 · Abstract (English)
Vision-Language-Action (VLA) models are a powerful paradigm for generalist robotic control. However, their high computational cost and limited control frequency hinder real-time robotic manipulation, especially when large vision-language backbones and iterative action heads run at every control step. Existing VLA acceleration methods often optimize individual components or rely on fixed acceleration rules, treating different control steps with largely fixed computation and overlooking the non-uniform reasoning demands of sequential embodied control. Inspired by human motor control, where cognitive and feedback resources concentrate on goal-sensitive stages, we argue that VLA models should learn when to invest full computation and when to reuse prior computation. We propose ElegantVLA, a plug-in phase-adaptive inference framework that accelerates VLA models through intra-model dynamic compute scheduling. ElegantVLA introduces a lightweight scheduler that observes temporal representation similarity, robot-motion cues, and episode progress to jointly allocate computation across the vision encoder, LLM, and action head. For perception-language reasoning, the scheduler selects a five-level Vision-LLM compute mode, from full recomputation to multi-step temporal reuse, based on visual-language representation stability. For action generation, it selects a three-level denoising mode, reusing intermediate denoising states during stable motion while preserving full refinement for goal-sensitive stages. By coordinating these decisions, ElegantVLA offers a general acceleration framework for modern VLA pipelines with explicit action-generation modules, without modifying or retraining the base model. Experiments on GR00T and CogACT achieve up to 2.55x and 3.77x speedup, and on six real-world GR00T tasks ElegantVLA cuts computation by 2.18x while raising control frequency from 13.8 Hz to 26.3 Hz.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。