提出多粒度自监督代码搜索框架,提升细粒度代码检索效果
MGS3: A Multi-Granularity Self-Supervised Code Search Framework
- 构建分层表示模块,融合语法结构实现多粒度代码表征
- 在536K+数据对上实现跨粒度检索性能显著提升
- 兼容现有预训练模型,适合代码复用与开发工具研发
为提升软件复用性与开发效率,代码搜索成为关键方向,旨在根据自然语言查询检索相关代码片段。尽管自监督代码预训练已取得进展,现有方法多聚焦于函数级代码与自然语言的对比学习,忽略了函数内丰富的块级与语句级细粒度代码片段,导致各粒度下性能不佳。为此,我们首先构建包含536,000+对自然语言与代码片段的多粒度代码搜索数据集MGCodeSearchNet。随后提出多粒度自监督对比学习代码搜索框架MGS³。该框架包含层次化多粒度表示模块(HMGR),利用语法结构关系进行分层表征,并将细粒度信息聚合至粗粒度表示中。在对比学习阶段,为细粒度代码构造同粒度正样本,并引入函数内负样本。在多个粒度级别的代码搜索基准上进行大量实验,结果表明该框架在多粒度代码搜索任务中表现优异,且具有模型无关性,可与现有预训练代码表征模型兼容。
原文摘要 · Abstract (English)
In the pursuit of enhancing software reusability and developer productivity, code search has emerged as a key area, aimed at retrieving code snippets relevant to functionalities based on natural language queries. Despite significant progress in self-supervised code pre-training utilizing the vast amount of code data in repositories, existing methods have primarily focused on leveraging contrastive learning to align natural language with function-level code snippets. These studies have overlooked the abundance of fine-grained (such as block-level and statement-level) code snippets prevalent within the function-level code snippets, which results in suboptimal performance across all levels of granularity. To address this problem, we first construct a multi-granularity code search dataset called MGCodeSearchNet, which contains 536K+ pairs of natural language and code snippets. Subsequently, we introduce a novel Multi-Granularity Self-Supervised contrastive learning code Search framework (MGS$^{3}$}). First, MGS$^{3}$ features a Hierarchical Multi-Granularity Representation module (HMGR), which leverages syntactic structural relationships for hierarchical representation and aggregates fine-grained information into coarser-grained representations. Then, during the contrastive learning phase, we endeavor to construct positive samples of the same granularity for fine-grained code, and introduce in-function negative samples for fine-grained code. Finally, we conduct extensive experiments on code search benchmarks across various granularities, demonstrating that the framework exhibits outstanding performance in code search tasks of multiple granularities. These experiments also showcase its model-agnostic nature and compatibility with existing pre-trained code representation models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。