CodeXEmbed提升多语言代码检索，7B模型性能超前代20%以上。
CodeXEmbed: A Generalist Embedding Model Family for Multiligual and Multi-task Code Retrieval
- 统一多语言代码与任务，构建通用检索框架。
- 7B模型在CoIR上超越Voyage-Code超20%，达新SOTA。
- 兼具代码与文本检索能力，适合研发与RAG应用。
尽管自然语言检索取得成功，代码检索仍属未充分探索领域。现有系统多针对自然语言设计，忽视代码特有挑战，难以应对多编程语言与跨领域任务的多样性。为此，我们提出CodeXEmbed，一个参数规模从4亿到70亿不等的大规模代码嵌入模型家族。通过创新训练流程，统一多种编程语言，并将各类代码相关任务转化为统一检索框架，显著提升模型泛化性与检索性能。7B模型在CoIR基准上超越此前领先模型Voyage-Code超过20%，达到新SOTA。此外，模型在广泛使用的BeIR文本检索基准上表现优异，展现跨领域适应性。实验表明，检索性能提升能显著增强代码相关任务中检索增强生成（RAG）的效果。
原文摘要 · Abstract (English)
Despite the success of text retrieval in many NLP tasks, code retrieval remains a largely underexplored area. Most text retrieval systems are tailored for natural language queries, often neglecting the specific challenges of retrieving code. This gap leaves existing models unable to effectively capture the diversity of programming languages and tasks across different domains, highlighting the need for more focused research in code retrieval. To address this, we introduce CodeXEmbed, a family of large-scale code embedding models ranging from 400M to 7B parameters. Our novel training pipeline unifies multiple programming languages and transforms various code-related tasks into a common retrieval framework, enhancing model generalizability and retrieval performance. Our 7B model sets a new state-of-the-art (SOTA) in code retrieval, outperforming the previous leading model, Voyage-Code, by over 20% on CoIR benchmark. In addition to excelling in code retrieval, our models demonstrate competitive performance on the widely adopted BeIR text retrieval benchmark, offering versatility across domains. Experimental results demonstrate that improving retrieval performance significantly enhances end-to-end Retrieval-Augmented Generation (RAG) performance for code-related tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。