用纯视觉变压器提升车载激光雷达语义分割性能
Vanilla ViT for Automotive Point Cloud Semantic Segmentation
- 采用无层级的ViT架构，配合精心设计的分词器和轻量解码头
- 在nuScenes等数据集上达到或超越当前最佳水平
- 适合追求简洁高效点云分割方案的研究者与工程师
纯视觉变压器（Vanilla Transformers）已成为处理文本、音频、图像和视频的主流架构，为多模态学习提供统一骨干网络。然而，当前点云语义分割的先进方法仍以卷积与局部/窗口注意力交替的U-Net架构为主。本文提出一种名为VaViT的方法，有效利用非层级的纯视觉变压器进行大规模车载激光雷达场景分割。通过设计合理的分词器、轻量级解码头及定制化数据增强策略，成功缩小了性能差距。所提方法在nuScenes、SemanticKITTI和Waymo Open Dataset上进行了充分验证，性能达到或超过现有最优方法，同时保持了ViT架构的简洁性。代码与模型已开源：https://github.com/valeoai/VaViT。
原文摘要 · Abstract (English)
Plain Transformers have become the de-facto architecture for processing text, audio, image, and video, offering a unified backbone for multimodal learning. However, state-of-the-art architectures for point cloud semantic segmentation remain dominated by U-Nets architectures where convolutions are interleaved with local or windowed attentions. In this work, we show how to effectively leverage vanilla, non-hierarchical ViTs for segmentation of large-scale automotive lidar scenes. We bridge the performance gap thanks to a carefully designed tokenizer, a lightweight decoder segmentation head, and tailored data augmentations. Our approach, VaViT for Vanilla ViT, matches or exceeds the performance of state-of-the-art methods while maintaining the simplicity of ViT architecture. We provide extensive evaluations on nuScenes, SemanticKITTI, and Waymo Open Dataset to validate the efficiency of our method. Code and models are available at https://github.com/valeoai/VaViT.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。