用检索增强生成技术实现零样本道路标志识别，无需训练即可准确识图。
SignRAG: A Retrieval-Augmented System for Scalable Zero-Shot Road Sign Recognition
- 先用视觉语言模型生成标志文字描述，再从向量库中检索候选标志
- 在理想图像上达到95.58%准确率，真实道路数据上达82.45%
- 适合需要快速部署、无标注数据的智能交通系统应用
自动驾驶中的道路标志识别对智能交通系统至关重要，但传统深度学习方法难以应对标志类别繁多及构建完整标注数据集的不现实性。本文提出一种新型零样本识别框架，将检索增强生成（RAG）范式应用于该任务。首先利用视觉语言模型（VLM）从输入图像生成标志的文本描述，再基于该描述从参考设计的向量数据库中检索最相关的候选标志。随后，大型语言模型（LLM）对检索结果进行推理，完成最终细粒度识别。我们在俄亥俄州MUTCD的303类法规标志上验证该方法，实验结果表明，该框架在理想参考图像上达到95.58%准确率，在挑战性真实道路数据上达到82.45%。本工作证明了基于RAG的架构可在无需特定任务训练的情况下，实现可扩展且高精度的道路标志识别系统。
原文摘要 · Abstract (English)
Automated road sign recognition is a critical task for intelligent transportation systems, but traditional deep learning methods struggle with the sheer number of sign classes and the impracticality of creating exhaustive labeled datasets. This paper introduces a novel zero-shot recognition framework that adapts the Retrieval-Augmented Generation (RAG) paradigm to address this challenge. Our method first uses a Vision Language Model (VLM) to generate a textual description of a sign from an input image. This description is used to retrieve a small set of the most relevant sign candidates from a vector database of reference designs. Subsequently, a Large Language Model (LLM) reasons over the retrieved candidates to make a final, fine-grained recognition. We validate this approach on a comprehensive set of 303 regulatory signs from the Ohio MUTCD. Experimental results demonstrate the framework's effectiveness, achieving 95.58% accuracy on ideal reference images and 82.45% on challenging real-world road data. This work demonstrates the viability of RAG-based architectures for creating scalable and accurate systems for road sign recognition without task-specific training.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。