用导航预判提前退出，让视觉语言模型更快响应自动驾驶场景。
Nav-EE: Navigation-Guided Early Exiting for Efficient Vision-Language Models in Autonomous Driving
- 根据导航信息预计算任务专属退出层，动态选择推理路径。
- 在CODA、Waymo等数据集上延迟降低63.9%，精度几乎不变。
- 适合需要低延迟的自动驾驶系统，尤其复杂路况决策场景。
视觉语言模型（VLMs）在自动驾驶中用于统一感知与推理，但高推理延迟阻碍实时部署。早退机制通过在中间层终止推理降低延迟，但其任务依赖性限制了跨场景泛化能力。我们发现，这一局限与自动驾驶特性一致：导航系统可预判未来环境（如交叉口、红绿灯），提示将执行的任务类型。为此提出Nav-EE框架，离线预计算各任务对应的退出层，并在线根据导航先验动态应用。在CODA、Waymo和BOSCH数据集上的实验表明，Nav-EE实现与全模型相当的精度，同时延迟最高降低63.9%。真实车辆集成使用Autoware Universe验证，推理延迟从600ms降至300ms，支持复杂场景下更快决策。结果表明，将导航前瞻与早退结合是大型模型高效部署于自动驾驶系统的可行路径。代码与数据见匿名仓库：https://anonymous.4open.science/r/Nav-EE-BBC4
原文摘要 · Abstract (English)
Vision-Language Models (VLMs) are increasingly applied in autonomous driving for unified perception and reasoning, but high inference latency hinders real-time deployment. Early-exit reduces latency by terminating inference at intermediate layers, yet its task-dependent nature limits generalization across diverse scenarios. We observe that this limitation aligns with autonomous driving: navigation systems can anticipate upcoming contexts (e.g., intersections, traffic lights), indicating which tasks will be required. We propose Nav-EE, a navigation-guided early-exit framework that precomputes task-specific exit layers offline and dynamically applies them online based on navigation priors. Experiments on CODA, Waymo, and BOSCH show that Nav-EE achieves accuracy comparable to full inference while reducing latency by up to 63.9%. Real-vehicle integration with Autoware Universe further demonstrates reduced inference latency (600ms to 300ms), supporting faster decision-making in complex scenarios. These results suggest that coupling navigation foresight with early-exit offers a viable path toward efficient deployment of large models in autonomous systems. Code and data are available at our anonymous repository: https://anonymous.4open.science/r/Nav-EE-BBC4
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。