用视觉RAG实现少样本细粒度商品分类，新商品无需重训练
A Visual RAG Pipeline for Few-Shot Fine-Grained Product Classification
- 结合视觉大模型与RAG，通过数据库新增样例支持新商品识别
- 在多源广告数据上达到86.8%准确率，可同时预测品类、价格与折扣
- 适合需要快速适应新品的零售自动化场景，如价格监控与推荐
尽管学习与计算机视觉算法迅速发展，细粒度分类（FGC）在众多实际应用中仍是开放性问题。例如，在零售领域，对快速变化且外观高度相似的商品及其属性进行识别，是实现自动价格监控与产品推荐的关键。本文提出一种新型视觉RAG流程，融合检索增强生成（RAG）与视觉语言模型（VLMs），用于少样本细粒度商品分类。该流程从多个零售商的宣传册中提取商品与促销信息，并同步预测细粒度商品编号、价格及折扣信息。相比以往方法，该视觉RAG流程的核心优势在于：无需重新训练即可预测新商品，仅需将少量类别样本加入RAG数据库即可。在GPT-4o、GPT-4o-mini与Gemini 2.0 Flash等多种VLM后端的对比中，本方法在多样化数据集上达到86.8%的准确率。
原文摘要 · Abstract (English)
Despite the rapid evolution of learning and computer vision algorithms, Fine-Grained Classification (FGC) still poses an open problem in many practically relevant applications. In the retail domain, for example, the identification of fast changing and visually highly similar products and their properties are key to automated price-monitoring and product recommendation. This paper presents a novel Visual RAG pipeline that combines the Retrieval Augmented Generation (RAG) approach and Vision Language Models (VLMs) for few-shot FGC. This Visual RAG pipeline extracts product and promotion data in advertisement leaflets from various retailers and simultaneously predicts fine-grained product ids along with price and discount information. Compared to previous approaches, the key characteristic of the Visual RAG pipeline is that it allows the prediction of novel products without re-training, simply by adding a few class samples to the RAG database. Comparing several VLM back-ends like GPT-4o [23], GPT-4o-mini [24], and Gemini 2.0 Flash [10], our approach achieves 86.8% accuracy on a diverse dataset.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。