用预训练代码模型检测跨语言编程漏洞,效果优于单一语言工具。
Fine-Tuning Code Language Models to Detect Cross-Language Bugs
- 用多语言代码交互数据微调13个代码模型,提升跨语言漏洞识别能力。
- 统一代码模型(UniXcoder-base)F1达0.7407,小模型表现优于大模型。
- 增加数据量显著提效,注释对部分模型有正负双重影响。
多语言编程在单个项目中使用多种编程语言日益普遍,但会引入跨语言漏洞(CLBs),这些漏洞源于不同语言间的交互,难以被单一语言的漏洞检测工具发现。本文研究预训练代码语言模型(CodeLMs)在CLB检测中的潜力。我们开发了跨语言代码识别工具CLCFinder,构建了包含三种语言组合(Python-C/C++、Java-C/C++、Python-Java)和九种交互类型的CLB数据集。对13个CodeLMs进行微调并评估性能,分析了数据集规模、词元序列长度和代码注释的影响。结果表明,所有13个CodeLMs在微调后均表现出不同程度的性能提升,其中UniXcoder-base取得最高F1分数(0.7407)。在实验设置下,小型CodeLMs表现优于大型模型。在单语言漏洞数据集上微调的CodeLMs在CLB检测中表现不佳,说明CLBs与单语言漏洞存在本质差异。增加微调数据集规模可显著提升性能,而更长的词元序列未必带来性能改善。代码注释的影响因模型而异,部分模型性能提升,部分则下降。
原文摘要 · Abstract (English)
Multilingual programming, which involves using multiple programming languages (PLs) in a single project, is increasingly common due to its benefits. However, it introduces cross-language bugs (CLBs), which arise from interactions between different PLs and are difficult to detect by single-language bug detection tools. This paper investigates the potential of pre-trained code language models (CodeLMs) in CLB detection. We developed CLCFinder, a cross-language code identification tool, and constructed a CLB dataset involving three PL combinations (Python-C/C++, Java-C/C++, and Python-Java) with nine interaction types. We fine-tuned 13 CodeLMs on this dataset and evaluated their performance, analyzing the effects of dataset size, token sequence length, and code comments. Results show that all 13 CodeLMs exhibited varying degrees of performance improvement after fine-tuning, with UniXcoder-base achieving the best F1 score (0.7407). Notably, within our experimental setup, small CodeLMs tended to performe better than large ones. CodeLMs fine-tuned on single-language bug datasets performed poorly on CLB detection, demonstrating the distinction between CLBs and single-language bugs. Additionally, increasing the fine-tuning dataset size significantly improved performance, while longer token sequences did not necessarily improve the model performance. The impact of code comments varied across models. Some fine-tuned CodeLMs' performance was improved, while others showed degraded performance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。