arXiv:2607.14695cs.RO2026-07中稿 · ICML

让流式推理的视觉语言动作模型实时运行,速度提升2.58倍。

Reflex: Real-Time VLA Control through Streaming Inference

论文配图:Reflex: Real-Time VLA Control through Streaming Inference
图 1 · 摘自论文原文
  • 利用时间无关性将注意力上下文分块,实现每步恒定时间缓存更新。
  • 在LIBERO和Kinetix上达到50Hz稳定推理,反应延迟降低54%。
  • 适合需要低延迟实时控制的机器人应用,如工业自动化。

流匹配视觉语言动作(VLA)模型可实现精确连续控制,但其迭代去噪特性与实时机器人存在根本冲突:全局时间步注入会破坏键值缓存,迫使选择慢速的$O(N^2)$重计算或数学错误的缓存复用。本文提出 extbf{Reflex}框架,通过利用感知编码器对去噪循环的功能独立性(时间无关性),实现流式推理。Reflex将注意力上下文分为静态、滑动和动态三部分,支持$O(1)$增量缓存更新,同时保持固定输入下的全批次等效注意力输出。为确保高频连续推理下的稳定性,引入自适应RMSNorm(AdaRMSNorm),通过流相位门控防止BFloat16数值崩溃。进一步通过异步流水线解耦视觉编码与动作生成,并结合算子融合减少核函数开销。在LIBERO和Kinetix基准测试中,Reflex实现2.58倍推理加速,达到50Hz稳定流式输出,反应延迟最多降低54%,且无性能损失,支持高效部署。

原文摘要 · Abstract (English)

Flow matching Vision-Language-Action (VLA) models promise precise continuous control, but their iterative denoising nature introduces fundamental incompatibilities with real-time robotics: global timestep injection invalidates KV-caching, forcing a choice between slow $O(N^2)$ re-computation or mathematically incorrect cache reuse. We present \textbf{Reflex}, a framework that enables \textit{real-time streaming inference} for flow matching policies by exploiting the \textit{Timestep-Invariance Property} -- that perception encoders are functionally independent of the denoising loop. Reflex partitions the attention context into static, sliding, and dynamic regions, enabling $O(1)$ incremental cache updates while preserving full-batch-equivalent attention outputs for fixed inputs. To ensure stability under continuous high-frequency inference, we introduce \textit{AdaRMSNorm}, an adaptive normalization layer that prevents BFloat16 numerical collapse by gating on flow phase. We further maximize throughput through an \textit{async pipeline} that decouples visual encoding from action generation, combined with \textit{operator fusion} that reduces kernel overhead. On LIBERO and Kinetix benchmarks, Reflex achieves a 2.58$\times$ inference speedup and 50Hz stable streaming, reducing reaction latency by up to 54\% and enabling efficient deployment without performance degradation.

机器人流式推理VLA实时控制

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。