让视觉语言动作模型流式并行,显著降低延迟和中断。
StreamingVLA: Streaming Vision-Language-Action Model with Action Flow Matching and Adaptive Early Observation
- 用动作流匹配替代分块动作,实现生成与执行重叠
- 通过动作重要性自适应观测,使执行与观察重叠
- 在不降性能前提下,延迟提速2.4倍,中断减少6.5倍
视觉-语言-动作(VLA)模型在自然语言驱动的感知与控制任务中表现优异,但其高计算成本带来显著效率挑战,尤其在资源受限的边缘平台部署时。由于观察、动作生成和执行阶段必须顺序进行且需等待前序完成,系统常出现频繁暂停和高延迟。为此,本文系统分析快速流畅生成的瓶颈,提出一种流式并行的VLA架构。首先,摒弃动作分块机制,采用动作流匹配,学习动作轨迹而非逐块去噪,实现动作生成与执行的重叠。其次,设计基于动作显著性的自适应观测机制,使执行与观测阶段重叠。无需牺牲性能,StreamingVLA实现了显著加速,达成2.4倍的延迟提速,并将执行中断次数降低6.5倍。
原文摘要 · Abstract (English)
Vision-language-action (VLA) models have demonstrated exceptional performance in natural language-driven perception and control. However, the high computational cost of VLA models poses significant efficiency challenges, particularly for resource-constrained edge platforms in real-world deployments. However, since different stages of VLA (observation, action generation and execution) must proceed sequentially, and wait for the completion of the preceding stage, the system suffers from frequent halting and high latency. To address this, We conduct a systematic analysis to identify the challenges for fast and fluent generation, and propose enabling VLAs with the ability to asynchronously parallelize across VLA stages in a "streaming" manner. First, we eliminate the reliance on action chunking and adopt action flow matching, which learns the trajectory of action flows rather than denoising chunk-wise actions. It overlaps the latency of action generation and execution. Second, we design an action saliency-aware adaptive observation mechanism, thereby overlapping the latency of execution and observation. Without sacrificing performance, StreamingVLA achieves substantial speedup and improves the fluency of execution. It achieves a 2.4 $\times$ latency speedup and reduces execution halting by 6.5 $\times$.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。