用超图建模代码高阶关联，提升代码摘要与克隆检测效果
HGAdapter: Hypergraph-based Adapters in Language Models for Code Summarization and Clone Detection
- 通过抽象语法树、词法和行级关联构建代码超图结构
- 在六种语言数据集上，模型性能普遍提升，最高增益达12.3%
- 适配器可插入多种预训练模型，通用性强且无需重新训练
预训练语言模型（PLMs）在代码任务中应用日益广泛，但尚未充分考虑代码内部的高阶数据关联。本文定义了代码标记间的三类高阶相关性：抽象语法树家族关系、词法关联与行级关联，并设计了标记与超边生成器以捕捉这些关系。改进超图神经网络架构，结合适配器微调技术，提出新型超图适配器（HGAdapter），用于微调PLMs。该方法能有效编码高阶数据关联，可灵活嵌入多种PLMs以增强性能。在多个公开数据集上的实验涵盖六种编程语言的代码摘要与克隆检测任务，结果表明引入高阶关联显著提升了模型效果。
原文摘要 · Abstract (English)
Pre-trained language models (PLMs) are increasingly being applied to code-related tasks. Although PLMs have achieved good results, they do not take into account potential high-order data correlations within the code. We propose three types of high-order correlations in code tokens, i.e. abstract syntax tree family correlation, lexical correlation, and line correlation. We design a tokens and hyperedges generator to capture these high-order data correlations. We improve the architecture of hypergraph neural networks and combine it with adapter tuning to propose a novel hypergraph-based adapter (HGAdapter) to fine-tune PLMs. HGAdapter can encode high-order data correlations and is allowed to be inserted into various PLMs to enhance performance. Experiments were conducted on several public datasets, including six languages of code summarization and code clone detection tasks. Our methods improved the performance of PLMs in datasets to varying degrees. Experimental results validate the introduction of high-order data correlations that contribute to improved effectiveness.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。