提出大规模评估嵌入压缩技术的框架CoRECT，解决压缩方法选择难题。
CoRECT: A Framework for Evaluating Embedding Compression Techniques at Scale
- 构建可控检索评估框架CoRECT，支持大规模压缩方法对比
- 非学习型压缩在1亿条文本上仍保持性能基本不变
- 适合需要高效压缩且关注检索质量的工程团队
稠密检索系统在多个基准上表现优异，但需占用大量内存存储大型搜索索引。近年来嵌入压缩技术显示可大幅缩减索引大小而几乎不影响排序质量。然而现有研究常忽略语料复杂性这一关键因素，近期工作表明语料规模和文档长度对稠密检索性能有显著影响。本文提出CoRECT（可控压缩技术检索评估框架），并配套构建新数据集集合，用于大规模评估嵌入压缩方法。通过基准测试八类代表性压缩方法，发现非学习型压缩在高达1亿条语料上仍能实现显著索引压缩，且性能损失统计不显著。但不同模型下最优压缩方法差异明显，凸显统一评估框架的重要性。所有代码、数据与结果已开源于GitHub与HuggingFace。
原文摘要 · Abstract (English)
Dense retrieval systems have proven to be effective across various benchmarks, but require substantial memory to store large search indices. Recent advances in embedding compression show that index sizes can be greatly reduced with minimal loss in ranking quality. However, existing studies often overlook the role of corpus complexity -- a critical factor, as recent work shows that both corpus size and document length strongly affect dense retrieval performance. In this paper, we introduce CoRECT (Controlled Retrieval Evaluation of Compression Techniques), a framework for large-scale evaluation of embedding compression methods, supported by a newly curated dataset collection. To demonstrate its utility, we benchmark eight representative types of compression methods. Notably, we show that non-learned compression achieves substantial index size reduction, even on up to 100M passages, with statistically insignificant performance loss. However, selecting the optimal compression method remains challenging, as performance varies across models. Such variability highlights the necessity of CoRECT to enable consistent comparison and informed selection of compression methods. All code, data, and results are available on GitHub and HuggingFace.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。