用文本检索增强生成，实现新车型零样本识别。
Zero-Shot Vehicle Model Recognition via Text-Based Retrieval-Augmented Generation
- 将车辆图像转为文本属性，通过文本检索匹配车型
- 相比CLIP基线提升近20%识别准确率
- 无需重训练，新增车型只需加文本描述
车辆品牌与型号识别（VMMR）是智能交通系统的重要任务，但现有方法难以适应新发布车型。对比语言-图像预训练（CLIP）虽具备强视觉-文本对齐能力，但固定预训练权重限制性能，需高昂代价进行图像特定微调。本文提出一种结合视觉语言模型（VLM）与检索增强生成（RAG）的流水线，支持基于文本推理的零样本识别。VLM将车辆图像转换为描述性属性，与文本特征数据库比对，检索相关条目并与描述拼接成提示，由语言模型（LM）推断品牌与型号。该设计避免大规模重训练，通过添加新车的文本描述即可快速更新。实验表明，该方法相较CLIP基线提升近20%识别准确率，展示了RAG增强的LM推理在智慧城市场景下可扩展车辆识别的潜力。
原文摘要 · Abstract (English)
Vehicle make and model recognition (VMMR) is an important task in intelligent transportation systems, but existing approaches struggle to adapt to newly released models. Contrastive Language-Image Pretraining (CLIP) provides strong visual-text alignment, yet its fixed pretrained weights limit performance without costly image-specific finetuning. We propose a pipeline that integrates vision language models (VLMs) with Retrieval-Augmented Generation (RAG) to support zero-shot recognition through text-based reasoning. A VLM converts vehicle images into descriptive attributes, which are compared against a database of textual features. Relevant entries are retrieved and combined with the description to form a prompt, and a language model (LM) infers the make and model. This design avoids large-scale retraining and enables rapid updates by adding textual descriptions of new vehicles. Experiments show that the proposed method improves recognition by nearly 20% over the CLIP baseline, demonstrating the potential of RAG-enhanced LM reasoning for scalable VMMR in smart-city applications.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。