根据查询动态调整检索维度,提速省资源。
AdaWidth: Query-Adaptive Embedding Width for Dense Retrieval

- 按查询需求自适应评估嵌入维度,共享前缀表示
- 仅对排名可能变化的查询扩展维度,减少40%~84%计算量
- 适用于大模型检索场景,提升效率且保持精度
高维嵌入是密集检索的核心,但并非所有维度在检索时都需计算。现有方法要么统一截断相同前导维度,要么为每查询掩码不同子集但仍存储完整嵌入。然而同一任务中查询所需维度差异显著。本文提出AdaWidth,根据每个查询的共享前缀表示自适应调整评估维度。通过正交前缀适配器对查询与文档统一旋转,将判别性信号集中于前导坐标,同时保持全维内积不变。轻量级路由器读取已有排名的顺序统计量,仅对排名可能变化的查询增加维度评估。进一步推导出前缀充分性分析:所需维度由召回截止处的竞争文档决定——随语料库大小对数增长,随检索深度对数下降,且跨查询呈重尾分布。在六个检索任务和五个冻结编码器上,AdaWidth在使用55%至84%更少维度的情况下,达到顶尖降维方法的NDCG@10性能。
原文摘要 · Abstract (English)
High-dimensional embeddings are central to dense retrieval, but not all of these dimensions need to be evaluated at retrieval time. Existing methods reduce dimensions in two ways: truncating the same leading dimensions for every query, or masking a different subset for each query while still storing and accessing the full embedding. Yet queries within a single task differ widely in the number of dimensions they need for their rankings to stabilize. We introduce AdaWidth, which adapts the number of evaluated dimensions to each query within a shared prefix representation. An orthogonal prefix adapter applies a single learned rotation to queries and documents alike, concentrating discriminative signal in leading coordinates while leaving every full width inner product unchanged. A lightweight router then reads order statistics off the ranking a query has already produced, and evaluates more dimensions only for the queries whose top results would change. We further derive a prefix sufficiency analysis showing that the required number of dimensions is set by the competing documents at the retrieval cutoff: it grows logarithmically with corpus size, decreases logarithmically with retrieval depth, and remains heavy-tailed across queries. Across six retrieval tasks and five frozen encoders, AdaWidth matches the NDCG@10 of state-of-the-art dimensionality reduction using 55% to 84% fewer dimensions per query.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。