arXiv:2506.07725cs.CVcs.AI2025-06ICCV被引 13

让大模型提前思考,实现自动驾驶的快速响应与高精度决策。

ETA: Efficiency through Thinking Ahead, A Dual Approach to Self-Driving with Large Models

  • 大模型提前处理过去帧数据,批量计算未来帧推理结果。
  • 在50毫秒延迟下达成69.53分驾驶得分,比现有方法提升8%。
  • 适合追求高精度与低延迟平衡的自动驾驶系统研发者。

如何在不牺牲推理速度的前提下利用大模型提升自动驾驶性能?现有方案多采用双系统架构:小模型负责实时反应,大模型进行深度分析。但当前方法仍难以对每一帧实现及时响应。本文提出一种异步系统ETA,核心思想是将当前帧的复杂计算转移到前序时间步完成,通过批量推理实现大模型对每一帧的快速响应。具体包括:(1)利用大模型对未来帧的预测,将信息特征从前向后传播至当前帧;(2)用小模型提取当前帧特征以保障实时性;(3)通过动作掩码机制融合双源特征,聚焦关键区域。在Bench2Drive CARLA Leaderboard-v2上,ETA以50毫秒近实时速度达到69.53分驾驶得分,较现有最优方法提升8%。

原文摘要 · Abstract (English)

How can we benefit from large models without sacrificing inference speed, a common dilemma in self-driving systems? A prevalent solution is a dual-system architecture, employing a small model for rapid, reactive decisions and a larger model for slower but more informative analyses. Existing dual-system designs often implement parallel architectures where inference is either directly conducted using the large model at each current frame or retrieved from previously stored inference results. However, these works still struggle to enable large models for a timely response to every online frame. Our key insight is to shift intensive computations of the current frame to previous time steps and perform a batch inference of multiple time steps to make large models respond promptly to each time step. To achieve the shifting, we introduce Efficiency through Thinking Ahead (ETA), an asynchronous system designed to: (1) propagate informative features from the past to the current frame using future predictions from the large model, (2) extract current frame features using a small model for real-time responsiveness, and (3) integrate these dual features via an action mask mechanism that emphasizes action-critical image regions. Evaluated on the Bench2Drive CARLA Leaderboard-v2 benchmark, ETA advances state-of-the-art performance by 8% with a driving score of 69.53 while maintaining a near-real-time inference speed at 50 ms.

自动驾驶大模型推理加速双系统

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。