评测大模型在代码检索中的召回效果，发现其在超大规模代码库上表现受限。
Recall Before Rerank: Benchmarking Deep Learning Models for Large-Scale Code-to-Code Retrieval
- 用大模型做代码归一化和查询改写，提升低性能模型精度
- 在千兆字节级代码库中，现有模型精度与可扩展性均显不足
- 适合关注代码检索系统效率与可扩展性的开发者参考
语义代码搜索与克隆检测对软件开发、维护和复用至关重要。本文评估了当前深度学习模型在大规模代码-代码搜索引擎第一阶段召回中的有效性、效率与可扩展性。在多种编程语言和数据集上的基准测试显示，这些模型在千兆字节级源代码集合中存在显著的精度与可扩展性瓶颈。我们提出基于大语言模型的代码归一化与查询重写方案，显著提升了低性能模型的精度。结果质疑了资源受限部署的可持续性以及当前专用代码大模型在不同数据集间的鲁棒性。最后，我们提出了构建可扩展、高效代码检索系统的可行建议。
原文摘要 · Abstract (English)
Semantic code search and clone detection are essential for software development, maintenance, and reuse. This paper evaluates the effectiveness, efficiency, and scalability of contemporary deep learning models for first-stage recall in large-scale code-to-code search engines. Benchmarking across multiple programming languages and datasets reveals critical limits in the precision and scalability of these models on Terabyte-scale source-code collections. We present LLM-based code normalisation and query-rewriting schemes that yield significant gains in precision for lower-performing models. Our results question the sustainability of resource-constrained deployment and the assumed robustness of current code-specialised LLMs across datasets. We conclude with actionable insights for building scalable, efficient code-retrieval systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。