arXiv:2608.15437cs.ROcs.CV2026-08

MM-BEV按重要性分层计算,实时降低自动驾驶感知延迟。

MM-BEV: Enhancing Timeliness by Computing Where and When it Matters

论文配图:MM-BEV: Enhancing Timeliness by Computing Where and When it Matters
图 1 · 摘自论文原文
  • 按安全关键区域和碰撞时间动态划分计算任务
  • 在nuScenes上推理延迟降1.96倍,端到端延迟降2.93倍
  • 适合资源受限的实车自动驾驶系统部署

多模态鸟瞰图(BEV)感知融合激光雷达深度精度与相机密集语义,但高计算成本和不理想的传感条件使其难以实现实时部署。现有方法主要压缩单个检测器,忽略了三个优化机会:相机与激光雷达输入中的结构化稀疏性、模态间的时间错位,以及许多检测物体不影响规划器即时决策的事实。我们提出MM-BEV,一个由‘在关键位置和时间计算’这一简单原则驱动的实时多模态BEV系统。该系统将感知分为两类:一类是针对本车制动距离内且碰撞时间短(TTC)的安全关键物体的强制工作;另一类是较不紧急区域的可选工作。在计算资源紧张时,优先处理强制任务并减少或舍弃可选任务。MM-BEV集成四种机制:(1) 基于运动外推检测的临界性排序时间兴趣区域选择器;(2) 共享形状相机裁剪的稀疏、区域感知特征提取与自适应分辨率;(3) 根据场景动态和TTC调整激光雷达扫描、图像分辨率与关键帧的延迟感知协调器;(4) 解耦感知与推理的异步调度器,跳过过时帧。在nuScenes数据集上,MM-BEV将推理延迟降低1.96倍,端到端延迟降低2.93倍,几何关键召回率无损失,安全关键召回率仅下降0.2个百分点。在配备Ouster-128激光雷达、BEV相机和Jetson AGX Orin的Clearpath Husky A300上,平均延迟进一步降低2.11倍,验证了其在真实自动驾驶系统中的潜力。

原文摘要 · Abstract (English)

Multimodal bird's-eye-view (BEV) perception combines LiDAR depth accuracy with dense camera semantics, but its high computational cost and imperfect sensing conditions make real-time deployment challenging. Existing methods largely compress individual detectors and overlook three opportunities: structured sparsity within camera and LiDAR inputs, timing misalignment between modalities, and the fact that many detected objects do not affect the planner's immediate action. We present MM-BEV, a real-time multimodal BEV system guided by a simple principle: compute where and when it matters. MM-BEV divides perception into mandatory work for safety-critical objects within braking distance of the ego vehicle and with short time-to-collision (TTC), and optional work for less urgent regions. It prioritizes mandatory work and reduces or sheds optional work under tight compute budgets. MM-BEV integrates four mechanisms: (1) a criticality-ranked temporal ROI selector based on motion-extrapolated detections from prior frames; (2) sparse, ROI-aware feature extraction using shared-shape camera crops at context-adaptive resolution and ROI-aware LiDAR voxelization; (3) a latency-aware coordinator that adapts LiDAR sweeps, image resolution, and keyframes according to scene dynamics and TTC; and (4) an asynchronous scheduler that decouples sensing from inference and skips stale frames. On nuScenes, MM-BEV reduces inference latency by 1.96x and end-to-end latency by 2.93x, with no loss in geometry-critical recall and only a 0.2 percentage-point drop in safety-critical recall. On a Clearpath Husky A300 equipped with an Ouster-128 LiDAR, BEV cameras, and a Jetson AGX Orin, MM-BEV further reduces mean latency by 2.11x, demonstrating its potential for real-world autonomous systems.

自动驾驶多模态感知实时计算BEV

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。