arXiv:2507.23331cs.CV2025-07被引 3

用图文对比学习提升小目标交通标志识别,解决数据不均衡难题。

Contrastive Learning-Driven Traffic Sign Perception: Multi-Modal Fusion of Text and Vision

  • 分两阶段:先用改进YOLO检测小目标,再通过图文对比学习分类。
  • 在TT100K上达78.4% mAP,准确率91.8%,远超主流方法。
  • 适合自动驾驶感知系统研发者,尤其关注长尾分布场景的团队。

交通标志识别是自动驾驶感知系统的核心,直接影响环境认知与行车安全。现有技术面临两大挑战:一是数据集存在显著长尾分布,导致传统卷积网络对低频及分布外类别识别性能下降;二是真实场景中交通标志多为小目标且尺度变化大,难以提取多尺度特征。为此,我们提出一种结合开放词汇检测与跨模态学习的两阶段框架。检测阶段,采用集成可重参数化视觉-语言路径聚合网络(RepVL-PAN)与SPD-Conv模块的NanoVerse YOLO模型,增强小目标、多尺度特征提取能力;分类阶段,设计交通标志识别多模态对比学习模型(TSR-MCL),通过对比视觉变压器提取的视觉特征与基于规则的BERT生成的语义特征,学习鲁棒且频率无关的表示,有效缓解数据不平衡带来的类别混淆问题。在TT100K数据集上,该方法在全类别识别的长尾检测任务中达到78.4%的mAP,分类准确率达91.8%,召回率为88.9%,显著优于主流算法,在复杂开放世界场景中展现出卓越的准确率与泛化能力。

原文摘要 · Abstract (English)

Traffic sign recognition, as a core component of autonomous driving perception systems, directly influences vehicle environmental awareness and driving safety. Current technologies face two significant challenges: first, the traffic sign dataset exhibits a pronounced long-tail distribution, resulting in a substantial decline in recognition performance of traditional convolutional networks when processing low-frequency and out-of-distribution classes; second, traffic signs in real-world scenarios are predominantly small targets with significant scale variations, making it difficult to extract multi-scale features.To overcome these issues, we propose a novel two-stage framework combining open-vocabulary detection and cross-modal learning. For traffic sign detection, our NanoVerse YOLO model integrates a reparameterizable vision-language path aggregation network (RepVL-PAN) and an SPD-Conv module to specifically enhance feature extraction for small, multi-scale targets. For traffic sign classification, we designed a Traffic Sign Recognition Multimodal Contrastive Learning model (TSR-MCL). By contrasting visual features from a Vision Transformer with semantic features from a rule-based BERT, TSR-MCL learns robust, frequency-independent representations, effectively mitigating class confusion caused by data imbalance. On the TT100K dataset, our method achieves a state-of-the-art 78.4% mAP in the long-tail detection task for all-class recognition. The model also obtains 91.8% accuracy and 88.9% recall, significantly outperforming mainstream algorithms and demonstrating superior accuracy and generalization in complex, open-world scenarios.

交通标志多模态对比学习小目标检测

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。