用线性复杂度设计高效3D语义占位网络,实时推理速度提升20倍
OccRWKV: Rethinking Efficient 3D Semantic Occupancy Prediction with Linear Complexity

- 分枝设计分离语义、占位与特征融合,引入语义与几何专用RWKV模块
- 在SemanticKITTI上达25.1的mIoU,推理速度22.2 FPS,是最佳基线的20倍
- 利用真实3D占位稀疏性,在鸟瞰图空间高效融合,适合机器人实时导航
3D语义占位预测网络在重建三维场景的几何与语义结构方面表现出色,为机器人导航和自动驾驶系统提供关键信息。然而,由于密集网络结构带来的高计算开销,现有方法难以平衡精度与延迟。本文提出OccRWKV,一种受Receptance Weighted Key Value (RWKV) 启发的高效语义占位网络。该网络将语义、占位预测与特征融合分为独立分支,分别引入Sem-RWKV与Geo-RWKV模块,以捕捉长程依赖,学习领域特定表示(即语义与几何),从而提升预测精度。基于真实世界3D占位的稀疏特性,通过将特征投影至鸟瞰图(BEV)空间,并提出BEV-RWKV模块,实现高效特征增强与融合。该设计使模型在不牺牲性能的前提下达到22.2 FPS的实时推理速度。实验表明,OccRWKV在SemanticKITTI数据集上取得25.1的mIoU,较最优基线Co-Occ快20倍,适用于机器人实时部署以提升自主导航效率。代码与视频见项目主页:https://jmwang0117.github.io/OccRWKV/
原文摘要 · Abstract (English)
3D semantic occupancy prediction networks have demonstrated remarkable capabilities in reconstructing the geometric and semantic structure of 3D scenes, providing crucial information for robot navigation and autonomous driving systems. However, due to their large overhead from dense network structure designs, existing networks face challenges balancing accuracy and latency. In this paper, we introduce OccRWKV, an efficient semantic occupancy network inspired by Receptance Weighted Key Value (RWKV). OccRWKV separates semantics, occupancy prediction, and feature fusion into distinct branches, each incorporating Sem-RWKV and Geo-RWKV blocks. These blocks are designed to capture long-range dependencies, enabling the network to learn domain-specific representation (i.e., semantics and geometry), which enhances prediction accuracy. Leveraging the sparse nature of real-world 3D occupancy, we reduce computational overhead by projecting features into the bird's-eye view (BEV) space and propose a BEV-RWKV block for efficient feature enhancement and fusion. This enables real-time inference at 22.2 FPS without compromising performance. Experiments demonstrate that OccRWKV outperforms the state-of-the-art methods on the SemanticKITTI dataset, achieving a mIoU of 25.1 while being 20 times faster than the best baseline, Co-Occ, making it suitable for real-time deployment on robots to enhance autonomous navigation efficiency. Code and video are available on our project page: https://jmwang0117.github.io/OccRWKV/.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。