为机器人部署视觉语言动作模型，揭示硬件瓶颈并提升效率。
Characterizing Vision-Language-Action Models across XPUs: Constraints and Acceleration for On-Robot Deployment
- 跨硬件评测发现边缘设备在成本与能耗上优于旗舰显卡。
- 发现推理分两阶段：计算密集型视觉模型后接内存密集型动作模块。
- 提出新优化技术，在边缘芯片上提速6倍且不影响任务成功率。
视觉-语言-动作（VLA）模型有望实现通用机器人控制，但其在机器人上的部署受限于实时推理在严苛成本与能耗预算下的表现。以往评估多依赖桌面级显卡，忽略了异构边缘加速器（GPU/XPUs/NPUs）带来的权衡与机遇。本文通过模型-硬件协同表征，开展低成本VLA部署的系统性分析。首先构建跨加速器排行榜，评估不同模型-硬件组合在成本、能耗、时间（CET）下的表现，发现合理配置的边缘设备在满足控制速率要求的同时，比旗舰显卡更具成本与能耗优势。其次，通过深度剖析发现一致的两阶段推理模式：先为计算密集型视觉语言模型骨干，后为内存密集型动作专家，导致各阶段硬件利用率不均。最后，基于此洞察，提出DP-Cache与V-AEFusion，减少扩散冗余并支持异步流水线并行，在GPU上实现最高2.9倍加速，边缘NPU上达6倍，仅伴随微小成功率下降。示例排行榜网站见：https://vla-leaderboard-01.vercel.app/。
原文摘要 · Abstract (English)
Vision-Language-Action (VLA) models are promising for generalist robot control, but on-robot deployment is bottlenecked by real-time inference under tight cost and energy budgets. Most prior evaluations rely on desktop-grade GPUs, obscuring the trade-offs and opportunities offered by heterogeneous edge accelerators (GPUs/XPUs/NPUs). We present a systematic analysis for low-cost VLA deployment via model-hardware co-characterization. First, we build a cross-accelerator leaderboard and evaluate model-hardware pairs under CET (Cost, Energy, Time), showing that right-sized edge devices can be more cost-/energy-efficient than flagship GPUs while meeting control-rate constraints. Second, using in-depth profiling, we uncover a consistent two-phase inference pattern: a compute-bound VLM backbone followed by a memory-bound Action Expert, which induces phase-dependent underutilization and hardware inefficiency. Finally, guided by these insights, we propose DP-Cache and V-AEFusion to reduce diffusion redundancy and enable asynchronous pipeline parallelism, achieving up to 2.9x speedup on GPUs and 6x on edge NPUs with only marginal success degradation. The example leaderboard website is available at: https://vla-leaderboard-01.vercel.app/.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。