首个专为代码检索设计的稀疏学习模型，提升效率与精度。
On the Challenges and Opportunities of Learned Sparse Retrieval for Code
- 用学习型稀疏表示替代传统稠密向量，适配代码特性。
- 在MTEB Code上10亿参数内达到75.4分，80亿参数达79.0分。
- 支持毫秒级检索，适合高吞吐代码搜索系统。
大型代码库的检索是现代基于大模型的软件工程系统的关键组成部分。现有方法主要依赖稠密嵌入模型，而针对代码的学習稀疏检索（LSR）尚未被充分探索。由于子词碎片化、自然语言查询与代码之间的语义鸿沟、编程语言多样性及代码文档长度问题，将稀疏检索应用于代码面临挑战，可能影响稀疏性和延迟。我们提出SPLADE-Code，首个专为代码检索设计的大规模学习稀疏检索模型系列（参数量600M-8B）。尽管采用轻量级单阶段训练流程，SPLADE-Code在10亿参数以下的检索器中达到当前最优性能（MTEB Code上75.4分），在更大规模下也表现优异（8B参数时达79.0分）。我们证明学习型扩展令牌对弥合词汇与语义匹配至关重要，并通过延迟分析表明，该方法可在百万条目集合上实现亚毫秒级检索，仅轻微损失效果。
原文摘要 · Abstract (English)
Retrieval over large codebases is a key component of modern LLM-based software engineering systems. Existing approaches predominantly rely on dense embedding models, while learned sparse retrieval (LSR) remains largely unexplored for code. However, applying sparse retrieval to code is challenging due to subword fragmentation, semantic gaps between natural-language queries and code, diversity of programming languages and sub-tasks, and the length of code documents, which can harm sparsity and latency. We introduce SPLADE-Code, the first large-scale family of learned sparse retrieval models specialized for code retrieval (600M-8B parameters). Despite a lightweight one-stage training pipeline, SPLADE-Code achieves state-of-the-art performance among retrievers under 1B parameters (75.4 on MTEB Code) and competitive results at larger scales (79.0 with 8B). We show that learned expansion tokens are critical to bridge lexical and semantic matching, and provide a latency analysis showing that LSR enables sub-millisecond retrieval on a 1M-passage collection with little effectiveness loss.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。