用SAM将框标注转为像素级分割，解决自动驾驶数据标注难题
SAM-Enhanced Segmentation on Road Datasets: Balancing Critical Classes in Autonomous Driving
- 用SAM将边界框转换为语义掩码，生成密集标注
- 在复杂天气下达48.1% mIoU，罕见类占比不足1%仍有效
- 适配多传感器、可复现，助力自动驾驶分割研究
稠密语义分割对自动驾驶至关重要，但多数多模态数据集缺乏像素级标注。Zenseact Open Dataset（ZOD）提供丰富的多传感器数据，但仅含边界框标签，限制了其在分割研究中的应用。本文提出一种基于Segment Anything Model（SAM）的标注流程，将边界框转化为语义掩码，实现像素级标注。在该试点研究中，我们处理超过10万帧数据，并人工校验2300帧子集（接受率36%），建立可靠基线。利用这些标注，评估基于Transformer的CLFT与基于CNN的DeepLabV3+模型在多种天气条件下的表现，最高达48.1% mIoU（CLFT-Hybrid）。针对行人、骑行者和标志等极端类别（像素占比不足1%）的不平衡问题，我们探索专用于稀有类别的模型。进一步在Iseauto自动驾驶平台上验证，实现77.5% mIoU，表明SAM生成表征可通过双向迁移学习有效跨传感器配置迁移。所有代码与标注均已公开，支持可复现研究。
原文摘要 · Abstract (English)
Dense semantic segmentation is essential for autonomous driving, yet many multi-modal datasets lack pixel-level annotations. The Zenseact Open Dataset (ZOD) provides rich multi-sensor data but only bounding-box labels, limiting its use for segmentation research. Our primary contribution is a Segment Anything Model (SAM)-based annotation pipeline that produces dense, pixel-level annotations for ZOD by converting bounding boxes into semantic masks. In this pilot study, we process over 100,000 frames and manually curate a 2,300-frame subset (36% acceptance rate) to establish a reliable baseline. Using these annotations, we evaluate transformer-based CLFT and CNN-based DeepLabV3+ architectures across diverse weather conditions, achieving up to 48.1% mIoU with CLFT-Hybrid. To address extreme class imbalance, where pedestrians, cyclists, and signs constitute less than 1% of pixels, we explore specialized models targeting rare classes. We further validate the pipeline on the Iseauto autonomous-vehicle platform, achieving 77.5% mIoU, and show that SAM-derived representations transfer effectively across sensor configurations via bidirectional transfer learning. All code and annotations are released to support reproducible research.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。