不用训练就能生成高质量代码向量,提升代码克隆检测效率
LSem2Vec: A Simple yet Effective Two-Stage Approach for Source Code Embedding
- 先用大模型提取代码语义,再用句子嵌入模型转为向量
- 在三个语言数据集上超越五种主流无监督方法
- 无需微调,适合快速部署到各类代码分析任务
大型语言模型(LLM)的兴起显著推动了软件工程中的人工智能发展,源代码嵌入在代码克隆检测、聚类等任务中发挥关键作用。然而,现有代码嵌入方法(包括基于LLM的方法)通常依赖昂贵的有监督训练或领域适配微调。本文提出LSem2Vec(LLM提取的代码语义到向量嵌入),一种简单有效的两阶段代码嵌入方法,通过结合大语言模型与句子嵌入模型实现。具体而言,LSem2Vec利用LLM提取代码语义,再由句子嵌入模型生成向量表示。相比以往方法,该方案无需任务特定训练或微调,有效缓解了LLM输出中的错误信息问题。我们在涵盖多种编程语言的三个数据集上,采用不同LLM与句子嵌入模型进行了系列实验。结果表明,LSem2Vec在性能上显著优于五种前沿无监督方法,展现出在软件工程中实现高效、鲁棒代码嵌入的巨大潜力。
原文摘要 · Abstract (English)
The advent of large language models (LLMs) has significantly advanced artificial intelligence in software engineering, with source code embeddings playing a crucial role in tasks such as source code clone detection and source code clustering. However, existing methods for source code embedding, including those based on LLMs, often rely on costly supervised training or fine-tuning for domain adaptation. This paper proposes LSem2Vec (LLM-extracted code Semantics to Vector embedding), a simple yet effective two-stage approach to embedding source code by combining large language and sentence embedding models. Specifically, LSem2Vec leverages an LLM to extract the semantics of source code, and then uses a sentence embedding model to generate representation vectors. Compared with previous approaches, LSem2Vec eliminates the need for task-specific training or fine-tuning and effectively addresses erroneous information commonly found in LLM-generated outputs. To evaluate the performance of LSem2Vec, we conducted a series of experiments across three datasets spanning different programming languages, using various LLMs and sentence embedding models. The experimental results demonstrate the effectiveness and superiority of LSem2Vec over five state-of-the-art unsupervised methods. Our findings highlight the potential of LSem2Vec to advance the field of software engineering by providing robust and efficient solutions for source code embedding tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。