让100亿参数自动驾驶模型提速4.7倍,实现实时推理。
FlashDrive: Flash Vision-Language-Action Inference for Autonomous Driving

- 四阶段协同优化:帧间缓存复用、非自回归推测解码、动态步长计算。
- 端到端延迟从717毫秒降至151毫秒,精度几乎不变。
- 适合追求实时性与高精度的自动驾驶系统研发者。
视觉-语言-动作(VLA)模型有望实现自动驾驶的端到端推理,但其计算开销过高,难以满足实时控制需求。核心挑战在于四个串联阶段:视觉编码在重叠视频帧上浪费算力;语言模型预填充重复计算可复用的上下文;推理标记串行生成却熵值低;流匹配去噪对非均匀速度场施加均等算力。单独优化任一阶段无法根治问题。本文提出FlashDrive,一种算法-系统协同设计框架,同时解决四大瓶颈。关键洞见是每类瓶颈均有轻量级加速方案:时间重叠支持跨帧键值缓存复用;驾驶领域推理具有低每标记熵和强块内相关性,使非自回归扩散起草器在推测解码中高效;速度场结构呈现两端尖锐、中间平坦,支持自适应步长缓存,聚焦算力于关键区域。结合系统级CUDA图编译与核融合,技术叠加效应显著。应用于Alpamayo 1.5-10B(W4A8量化),端到端延迟从717毫秒降至151毫秒(4.7倍提升),精度基本不变:[email protected]仅变化0.08米,minADE1改善,仿真中闭环碰撞率与偏离道路率下降。单卡推理频率从1.4~Hz提升至6.6~Hz,显著推进端到端自动驾驶向实时部署迈进。
原文摘要 · Abstract (English)
Vision-Language-Action (VLA) models promise to bring end-to-end reasoning to autonomous driving, but their computational cost remains far too high for real-time control. The core challenge is structural: VLA inference is not a single bottleneck but a cascade of four. Visual encoding wastes compute on overlapping video frames; language-model prefill recomputes context that could be carried over from the previous timestep; reasoning tokens are generated serially despite low entropy; and flow-matching denoising applies uniform compute to a non-uniform velocity field. Addressing any one stage in isolation leaves the others untouched. We propose FlashDrive, an algorithm-system co-design framework that targets all four stages simultaneously. Our key insight is that each bottleneck admits a distinct, lightweight algorithmic shortcut: temporal overlap enables streaming KV-cache reuse across frames; the low per-token entropy and strong intra-block correlations of driving-domain reasoning make a non-autoregressive diffusion drafter highly effective for speculative decoding; and the velocity field's structure---sharp at the endpoints, flat in the middle---permits adaptive step caching that concentrates compute where it matters. Layered on system-level CUDA Graph compilation and kernel fusion, these techniques compound. Applied to Alpamayo 1.5-10B with W4A8 quantization, FlashDrive reduces end-to-end latency from 717ms to 151ms (4.7x) while leaving accuracy essentially unchanged: [email protected] shifts by only 0.08m, minADE1 improves, and closed-loop collision and off-road rates improve in simulation. By raising a 10B-parameter reasoning VLA from 1.4~Hz to 6.6~Hz on a single GPU, FlashDrive moves end-to-end autonomous driving substantially closer to real-time deployment.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。