通过状态感知的推测推理,实现高效可靠的机器人长序列动作预测。
Algorithm-Architecture Co-Design for Efficient VLA Inference via Speculative Inference and Verification

- 根据环境状态动态切换长序列推测与选择性验证
- 在LIBERO和ManiSkill上降低端到端延迟,成功率不变
- 适合需要实时、可靠动作决策的机器人系统
视觉-语言-动作(VLA)模型在具身智能领域表现出色,但高计算开销和有限的动作预测长度制约了实时部署。尽管已有专用加速器Dadu-Corki,却未利用机器人与环境的自然交互模式,导致预测动作长度较短。我们发现,机器人环境天然存在主动与非主动状态交替:主动状态下精确动作至关重要,非主动状态下动作影响较小。这一观察带来新调度机会:在非主动状态进行长动作序列推测,主动状态则选择性验证。为此提出SpecVLA,一种算法-系统协同设计框架,自适应平衡动作长度、推理延迟与任务可靠性。算法层面,引入状态感知的VLA执行范式,并通过差分残差与块级混合精度量化构建轻量级验证模型(sVLA)。系统层面,设计含GPU与机器人专用硬件模块的异构架构,采用推测数据流解耦VLA与sVLA并行执行。在OpenVLA和RDT上对LIBERO与ManiSkill基准的综合评估显示,SpecVLA显著降低端到端延迟,同时保持任务成功率。通过及时验证的长序列推测,SpecVLA实现了高效可靠的实时机器人操作。
原文摘要 · Abstract (English)
Vision-Language-Action (VLA) models have demonstrated remarkable capabilities in the field of embodied AI, but their high computational cost and limited predicted action length hinder real-time deployment. Although Dadu-Corki, a dedicated accelerator for efficient embodied AI, has been introduced, it does not exploit the inherent interaction patterns between the robot and its environment, which results in a relatively short predicted action length. We observe that robotic environments naturally alternate between active states-where precise actions are crucial-and inactive states-where actions have limited impact on task success. This insight enables a new scheduling opportunity: long-action-length speculative prediction in inactive states, paired with selective verification in active states. We propose SpecVLA, an algorithm-system co-design framework that adaptively balances action length, inference latency, and task reliability. On the algorithm side, SpecVLA introduces a state-aware VLA inference execution paradigm and a hardware-friendly construction of a smaller verification model (sVLA) using differential residuals and block-wise mixed-precision quantization. On the system side, we develop a heterogeneous architecture consisting of a GPU and a robotic-specific hardware module, along with a speculative dataflow that decouples VLA and sVLA through parallel execution. Comprehensive evaluations on OpenVLA and RDT across LIBERO and ManiSkill benchmarks show that SpecVLA reduces end-to-end latency significantly while preserving task success rate. By enabling long-action-length speculative prediction with timely verification, SpecVLA achieves real-time robotic manipulation with both high efficiency and reliability.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。