ColBERT-serve让小服务器高效跑检索模型，省90%内存
ColBERT-serve: Efficient Multi-Stage Memory-Mapped Scoring
- 用内存映射技术处理索引，大幅降低内存占用
- 多阶段混合打分，查询延迟降低，支持并发高负载
- 适合资源受限场景下部署大型检索模型
我们研究在资源有限的情况下，如何高效服务多个用户同时使用晚交互检索模型（如ColBERT）。针对索引无法完全加载到内存的问题，提出ColBERT-serve系统，采用内存映射策略，将内存占用减少90%，可在廉价服务器上部署；同时引入多阶段架构与混合打分机制，显著降低查询延迟，支持大量并发查询。该系统有效提升了大规模检索模型在实际部署中的可行性。
原文摘要 · Abstract (English)
We study serving retrieval models, specifically late interaction models like ColBERT, to many concurrent users at once and under a small budget, in which the index may not fit in memory. We present ColBERT-serve, a novel serving system that applies a memory-mapping strategy to the ColBERT index, reducing RAM usage by 90% and permitting its deployment on cheap servers, and incorporates a multi-stage architecture with hybrid scoring, reducing ColBERT's query latency and supporting many concurrent queries in parallel.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。