通过动态量化与中心点复用，加速视觉语言动作模型推理
A Motion-Aware Vector Quantization Framework with Centroid Reuse for Efficient VLA Inference
- 根据机器人执行状态动态调整量化精度，减少内存访问
- 提出融合中心点的向量化GEMM，通过空间聚合与时间复用消除冗余计算
- 适合需实时推理的具身智能系统，尤其在资源受限场景下优势明显
视觉-语言-动作（VLA）模型在具身智能中展现出巨大潜力，但其在GPU上的高推理延迟限制了实时部署。现有加速器如Dadu-Corki虽提升效率，却将VLA模型视为全精度负载，导致内存与计算中仍有大量冗余未被挖掘。本文提出VQVLA，一种算法-硬件协同设计框架，通过利用权重相似性与执行动态特性加速VLA推理。首先引入MotionVQ，一种运动感知的向量量化方案，根据机器人执行状态动态调整量化精度，降低内存访问同时保持任务成功率。随后提出合并中心点的向量化GEMM范式，基于码本索引表示运算，通过空间聚合与中心点的时间复用消除冗余乘法。为实现这些优化，设计了支持动态精度选择与中心点复用计算的加速器。实验表明，VQVLA相较A100 GPU、Dadu-Corki、LUT-DLA、CodeGEMM和ShiftAddLLM分别获得6.5倍、2.8倍、1.9倍、3.3倍和4.3倍加速，且精度损失可忽略。
原文摘要 · Abstract (English)
Vision-Language-Action (VLA) models have demonstrated strong potential for embodied AI, yet their high inference latency on GPUs limits real-time deployment. Existing accelerators, such as Dadu-Corki, improve efficiency but treat VLA models as full-precision workloads, leaving substantial redundancy in both memory and computation underexploited. In this paper, we propose VQVLA, an algorithm-hardware co-design framework that accelerates VLA inference by exploiting weight similarity and execution dynamics. We first introduce MotionVQ, a motion-aware vector quantization scheme that dynamically adjusts quantization precision based on the robot's execution state, reducing memory access while preserving task success rate. We then propose a merged-centroid vectorized GEMM paradigm that operates on the codebook-index representation, eliminating redundant multiplications through spatial aggregation and temporal reuse of centroids. To realize these optimizations, we design an accelerator that efficiently supports dynamic precision selection and centroid-reuse computation. Experimental results show that VQVLA achieves 6.5x, 2.8x, 1.9x, 3.3x, and 4.3x speedup over the A100 GPU, Dadu-Corki, LUT-DLA, CodeGEMM, and ShiftAddLLM, respectively, with negligible accuracy degradation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。