提出视觉瓶颈理论,用新指标诊断检索模型容量,提升商品搜索召回率
The Voronoi Bottleneck: Capacity-Aware Dense Retrieval for Product Search
- 基于几何瓶颈理论,构建可计算的容量利用率评分
- 在10万查询数据集上,召回率提升1.9个百分点(84.9 vs 83.0)
- 无需推理开销,可直接替换训练目标,适合部署优化
密集嵌入检索将所有相关性信息压缩到单一内积中,导致嵌入维度固定时存在根本性的几何限制——维诺伊瓶颈。本文提出三项贡献:(1) 统一容量理论,证明维诺伊复杂度与符号秩对顶1检索等价,得到紧致维度边界,并提出可计算的容量利用率评分(CUS),无需相关性标签即可以AUC > 0.8预测查询失败;(2) 诊断能力,CUS识别两种容量状态——中等(δ ≳ 1)时密度感知训练有效,真空(δ ≪ 1)时无效,为重训练提供事前判断;(3) DART训练方法,引入自适应温度密度加权对比损失(AT-DW-InfoNCE),推导出最优权重α* = 2.0。在包含10万查询的合成商品搜索数据集上,DART相较同数据集InfoNCE基线提升+1.9 Recall@100(84.9 ± 0.0 vs. 83.0 ± 0.3;8次种子实验,p < 0.001),优于焦点损失和温度调度方案。DART无需推理开销,是任意双编码器系统的即插即用训练目标。
原文摘要 · Abstract (English)
Dense embedding retrieval compresses all relevance information into a single inner product, imposing a fundamental geometric limit -- the Voronoi Bottleneck -- on the number of query-document relevance patterns expressible at fixed embedding dimension (d). We make three contributions. (1) Unified capacity theory. We prove that Voronoi complexity and sign-rank are equivalent for top-1 retrieval, yielding tight dimension bounds and a computable diagnostic, the Capacity Utilization Score (CUS), that predicts per-query retrieval failure with AUC (> 0.8) without relevance labels. (2) Diagnosis. CUS identifies two capacity regimes -- moderate ((δ\gtrsim 1)), where density-aware training yields measurable gains, and vacuous ((δ\ll 1)), where it does not -- giving practitioners an a priori check before investing in retraining. (3) DART training. We introduce AT-DW-InfoNCE, an Adaptive-Temperature Density-Weighted contrastive objective with formally derived optimal weighting (α^* = 2.0). On a 100K-query synthetic product-search corpus with controlled relevance structure, DART improves +1.9 Recall@100 over a same-data InfoNCE baseline ((84.9 \pm 0.0) vs. (83.0 \pm 0.3); 8 seeds, (p < 0.001)), outperforming focal loss and temperature-schedule alternatives. DART requires zero inference-time overhead -- it is a drop-in training objective that improves any dual-encoder system.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。