用几何方法精准剪枝，降低检索模型存储开销
A Voronoi Cell Formulation for Principled Token Pruning in Late-Interaction Retrieval Models
- 将词元影响范围建模为嵌入空间中的沃罗诺伊区域
- 在保持检索效果的同时，显著减少索引存储量
- 适合关注模型可解释性与高效部署的研究者
晚交互模型如ColBERT在多种检索任务中表现优异，但需为每个文档词元存储密集嵌入，造成巨大索引存储开销。以往工作尝试基于统计和经验度量剪枝低重要性词元嵌入，但常缺乏理论基础或效果不佳。本文提出一种基于超空间几何的框架，将词元剪枝问题转化为嵌入空间中的沃罗诺伊区域估计。通过将每个词元的影响视为其沃罗诺伊区域的度量，该方法实现了有理论依据的剪枝，在保留检索性能的同时减少索引大小。实验表明，该方法不仅是一种有效的剪枝策略，还可作为改进和解析密集检索系统中词元级行为的有力工具。
原文摘要 · Abstract (English)
Late-interaction models such as ColBERT offer competitive performance across various retrieval tasks but require storing a dense embedding for each document token, leading to a substantial index storage overhead. Past works address this by attempting to prune low-importance token embeddings based on statistical and empirical measures, but they often either lack formal grounding or are ineffective. To address these shortcomings, we introduce a framework grounded in hyperspace geometry and cast token pruning as a Voronoi cell estimation problem in the embedding space. By interpreting each token's influence as a measure of its Voronoi region, our approach enables principled pruning that retains retrieval quality while reducing index size. Through our experiments, we demonstrate that this approach serves not only as a competitive pruning strategy but also as a valuable tool for improving and interpreting token-level behavior within dense retrieval systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。