用低参数方法提升代码检索精度，训练快且效果显著。
LoRACode: LoRA Adapters for Code Embeddings
- 基于LoRA的轻量级适配器，仅需原模型2%参数即可微调。
- 在200万代码样本上25分钟完成训练，代码搜索MRR提升9.1%。
- 支持多语言任务适配，适合高效部署于代码检索场景。
代码嵌入对语义代码搜索至关重要，但现有方法难以捕捉代码的精确语法与上下文特征。开源模型如CodeBERT和UniXcoder存在可扩展性与效率问题，而高性能专有系统则带来高昂计算成本。本文提出一种基于低秩适应（LoRA）的参数高效微调方法，构建针对代码检索的任务特定适配器。该方法将可训练参数减少至基模型的2%以下，可在两块H100 GPU上25分钟内完成200万样本的快速微调。实验表明，代码到代码搜索的均倒数排名（MRR）最高提升9.1%，文本到代码搜索任务在多编程语言下最高达86.69%。任务与语言层面的差异化适配揭示了代码检索对语法与语言差异的敏感性。为促进研究，代码与预训练模型已公开。
原文摘要 · Abstract (English)
Code embeddings are essential for semantic code search; however, current approaches often struggle to capture the precise syntactic and contextual nuances inherent in code. Open-source models such as CodeBERT and UniXcoder exhibit limitations in scalability and efficiency, while high-performing proprietary systems impose substantial computational costs. We introduce a parameter-efficient fine-tuning method based on Low-Rank Adaptation (LoRA) to construct task-specific adapters for code retrieval. Our approach reduces the number of trainable parameters to less than two percent of the base model, enabling rapid fine-tuning on extensive code corpora (2 million samples in 25 minutes on two H100 GPUs). Experiments demonstrate an increase of up to 9.1% in Mean Reciprocal Rank (MRR) for Code2Code search, and up to 86.69% for Text2Code search tasks across multiple programming languages. Distinction in task-wise and language-wise adaptation helps explore the sensitivity of code retrieval for syntactical and linguistic variations. To foster research in this area, we make our code and pre-trained models publicly available.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。