评测190个开源视觉语言模型在商品检索中的表现，发现数据质量比模型大小更重要。
What Matters for Grocery Product Retrieval with Open Source Vision Language Models
- 对比不同预训练数据、架构和分辨率，评估开源模型在商品检索任务的表现。
- 数据过滤可提升16.6%准确率，超过参数翻倍的收益；小模型也能超越大模型。
- 尽管类别区分能力强，但对相似商品的排序仍存在17.5%的精度差距，适合零售系统优化者。
多模态商品检索（MPR）支撑无感结账与自动库存管理，但需精细识别商品型号（SKU），而主流视觉语言基准无法覆盖此需求。本文首次系统性地对190个开源视觉语言模型（VLMs）在GroceryVision挑战赛的MPR任务上进行零样本评估，分离预训练数据、模型架构与输入分辨率的影响。分析得出三个可操作结论：(1) 数据质量优于规模，从原始网络抓取切换至过滤数据集可带来最高16.6%的准确率提升，超过参数翻倍的增益；(2) 高效模型可胜出，MobileCLIP-B（150M参数）优于351M参数但在噪声数据上训练的模型；我们提出语义功率密度（ϕ）作为效率指标，惩罚低于阈值的准确率；(3) 精度鸿沟依然存在，当前最优模型在Recall@5达94.5%，但在Recall@1下降17.5%，表明对比嵌入能有效聚类品类，却难以区分视觉相似的SKU。代码与评估脚本已公开于https://github.com/upeee/openmpr。
原文摘要 · Abstract (English)
Multimodal product retrieval (MPR) underpins checkout-free retail and automated inventory systems, yet it demands fine-grained SKU discrimination that standard vision-language benchmarks fail to capture. We present the first systematic zero-shot evaluation of 190 open-source VLMs on the MPR task of the GroceryVision Challenge, isolating pre-training data, architecture, and input resolution. Our analysis yields three actionable findings. \textbf{(1) Data quality trumps scale.} Switching from raw web-scrapes to filtered datasets delivers up to 16.6\% accuracy gains, exceeding the benefit of doubling model parameters. \textbf{(2) Efficient models can win.} MobileCLIP-B (150M parameters) outperforms 351M counterparts trained on noisy data. We introduce \textit{semantic power density} ($ϕ$), an efficiency metric that penalizes sub-threshold accuracy. \textbf{(3) A precision gap persists.} State-of-the-art models achieve 94.5\% Recall@5 but suffer a 17.5\% drop at Recall@1, revealing that contrastive embeddings cluster categories effectively but fail to rank visually similar SKUs. Code and evaluation scripts are available at \url{https://github.com/upeee/openmpr}.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。