用词频统计提升图像检索可解释性，高效精准定位目标图
Visual Words Meet BM25: Sparse Auto-Encoder Visual Word Scoring for Image Retrieval
- 将BM25词频算法用于视觉词激活，通过稀疏编码提取关键视觉特征
- 在7个基准上召回率超99.3%，仅需重排200张图即接近稠密检索精度
- 结果可追溯到具体视觉词，适合需要可解释性的图像搜索场景
稠密图像检索虽准确但缺乏可解释性且计算开销大。本文提出BM25-V，将Okapi BM25评分应用于视觉变换器块特征的稀疏自编码器（SAE）生成的视觉词激活。在大规模图像库中，视觉词文档频率高度不平衡，呈类似齐普夫分布，使BM25的逆文档频率（IDF）权重能有效抑制常见低信息量词，突出稀有判别性词。BM25-V通过稀疏倒排索引操作实现高召回候选检索，作为高效第一阶段检索器用于稠密重排序。在七个基准测试中，BM25-V的Recall@200 ≥ 0.993，支持两阶段流程：每查询仅重排序K=200个候选，平均恢复近稠密检索精度的0.2%以内。一个在ImageNet-1K上训练一次的SAE可零样本迁移至七个细粒度任务无需微调，且BM25-V的检索决策可归因于特定视觉词，并量化其IDF贡献。
原文摘要 · Abstract (English)
Dense image retrieval is accurate but offers limited interpretability and attribution, and it can be compute-intensive at scale. We present \textbf{BM25-V}, which applies Okapi BM25 scoring to sparse visual-word activations from a Sparse Auto-Encoder (SAE) on Vision Transformer patch features. Across a large gallery, visual-word document frequencies are highly imbalanced and follow a Zipfian-like distribution, making BM25's inverse document frequency (IDF) weighting well suited for suppressing ubiquitous, low-information words and emphasizing rare, discriminative ones. BM25-V retrieves high-recall candidates via sparse inverted-index operations and serves as an efficient first-stage retriever for dense reranking. Across seven benchmarks, BM25-V achieves Recall@200 $\geq$ 0.993, enabling a two-stage pipeline that reranks only $K{=}200$ candidates per query and recovers near-dense accuracy within $0.2$\% on average. An SAE trained once on ImageNet-1K transfers zero-shot to seven fine-grained benchmarks without fine-tuning, and BM25-V retrieval decisions are attributable to specific visual words with quantified IDF contributions.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。