arXiv:2409.20166cs.CV2024-09ICRA被引 1

用SAM生成道路区域候选,结合CLIP优化筛选,提升自动驾驶道路检测精度。

Task-Oriented Pre-Training for Drivable Area Detection

  • 先用SAM生成大量道路候选区域,再用CLIP+特定类别增强微调筛选有效区域。
  • 在KITTI数据集上,模型性能显著优于无预训练及传统预训练方法。
  • 适合自动驾驶中道路检测任务,尤其在标注数据少时优势明显。

预训练在深度学习中至关重要,能提升模型在多种任务上的表现。传统方法依赖大规模数据和高算力,且难以捕捉深层任务相关特征。本文提出一种面向任务的预训练方法:首先使用分割一切模型(SAM)生成冗余的分割提议,再通过特定类别增强微调(SCEF)策略对对比语言-图像预训练(CLIP)模型进行微调,从中筛选与可行驶区域最相关的提议。该方法可生成大量粗粒度训练数据,再经人工标注数据进一步微调,从而提升模型性能。在KITTI道路数据集上的全面实验表明,该方法在各项指标上均优于未预训练模型,且超越传统预训练方法,达到当前自训练方法的最佳水平。

原文摘要 · Abstract (English)

Pre-training techniques play a crucial role in deep learning, enhancing models' performance across a variety of tasks. By initially training on large datasets and subsequently fine-tuning on task-specific data, pre-training provides a solid foundation for models, improving generalization abilities and accelerating convergence rates. This approach has seen significant success in the fields of natural language processing and computer vision. However, traditional pre-training methods necessitate large datasets and substantial computational resources, and they can only learn shared features through prolonged training and struggle to capture deeper, task-specific features. In this paper, we propose a task-oriented pre-training method that begins with generating redundant segmentation proposals using the Segment Anything (SAM) model. We then introduce a Specific Category Enhancement Fine-tuning (SCEF) strategy for fine-tuning the Contrastive Language-Image Pre-training (CLIP) model to select proposals most closely related to the drivable area from those generated by SAM. This approach can generate a lot of coarse training data for pre-training models, which are further fine-tuned using manually annotated data, thereby improving model's performance. Comprehensive experiments conducted on the KITTI road dataset demonstrate that our task-oriented pre-training method achieves an all-around performance improvement compared to models without pre-training. Moreover, our pre-training method not only surpasses traditional pre-training approach but also achieves the best performance compared to state-of-the-art self-training methods.

道路检测预训练CLIP自动驾驶

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。