将占用预测与3D检测高效融合，仅增1.1毫秒延迟
UltimateDO: An Efficient Framework to Marry Occupancy Prediction with 3D Object Detection via Channel2height
- 用2D卷积重构模型，实现轻量级任务协同
- 在nuScenes上同时达成高精度与低延迟
- 适合边缘设备部署，尤其看重效率的自动驾驶系统
占据预测与3D目标检测是现代自动驾驶系统的两项核心任务。为在多种边缘芯片上实现更高精度与更低时延的平衡，现有方法或采用独立模型，或设计多任务并行头结构，但常面临3D卷积、Transformer等模块部署困难或任务间协调不足的问题。为此，本文提出UltimateDO框架，通过通道到高度的重参数化（Channel2height），以2D卷积为基础重构模型，将FlashOcc中的轻量级占据预测头与3D检测网络深度融合，实现两者相互促进，额外延迟仅1.1ms。该方法在挑战性的nuScenes系列数据集上验证有效。
原文摘要 · Abstract (English)
Occupancy and 3D object detection are characterized as two standard tasks in modern autonomous driving system. In order to deploy them on a series of edge chips with better precision and time-consuming trade-off, contemporary approaches either deploy standalone models for individual tasks, or design a multi-task paradigm with separate heads. However, they might suffer from deployment difficulties (i.e., 3D convolution, transformer and so on) or deficiencies in task coordination. Instead, we argue that a favorable framework should be devised in pursuit of ease deployment on diverse chips and high precision with little time-consuming. Oriented at this, we revisit the paradigm for interaction between 3D object detection and occupancy prediction, reformulate the model with 2D convolution and prioritize the tasks such that each contributes to other. Thus, we propose a method to achieve fast 3D object detection and occupancy prediction (UltimateDO), wherein the light occupancy prediction head in FlashOcc is married to 3D object detection network, with negligible additional timeconsuming of only 1.1ms while facilitating each other. We instantiate UltimateDO on the challenging nuScenes-series benchmarks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。