用大模型自动修复代码漏洞,提升安全性与可靠性。
Enhancing Source Code Security with LLMs: Demystifying The Challenges and Generating Reliable Repairs
- 基于指令的系统,可自动识别、描述并修复漏洞代码。
- 相比其他大模型,安全修复准确率提升12%。
- 提供数据准备、模型选择与评估的实用指南,适合开发者参考。
随着人工智能计算的飞速发展,大语言模型(LLMs)的进步日新月异,但在安全领域仍缺乏清晰的指导规范。本文系统梳理了贯穿大模型全生命周期的三大技术挑战:(i) 数据收集与标注;(ii) 系统设计与学习;(iii) 性能评估。针对这些问题,提出 exttt{SecRepair}——一个基于指令的 LLM 系统,可可靠地识别、描述并自动修复存在漏洞的源代码。该系统配套提供数据准备与增强、主流 LLM 模型选型与适配、评估流程等可操作指南。 exttt{SecRepair} 采用基于语义奖励的强化学习微调方法,兼顾代码功能与安全性。实证分析表明,在强化学习训练下,其安全修复性能比其他 LLM 提升 12%。此外,通过自动化评估指标,在真实测试用例中验证了其生成的修复代码具有高可靠性、功能性与可编译性。
原文摘要 · Abstract (English)
With the recent unprecedented advancements in Artificial Intelligence (AI) computing, progress in Large Language Models (LLMs) is accelerating rapidly, presenting challenges in establishing clear guidelines, particularly in the field of security. That being said, we thoroughly identify and describe three main technical challenges in the security and software engineering literature that spans the entire LLM workflow, namely; \textbf{\textit{(i)}} Data Collection and Labeling; \textbf{\textit{(ii)}} System Design and Learning; and \textbf{\textit{(iii)}} Performance Evaluation. Building upon these challenges, this paper introduces \texttt{SecRepair}, an instruction-based LLM system designed to reliably \textit{identify}, \textit{describe}, and automatically \textit{repair} vulnerable source code. Our system is accompanied by a list of actionable guides on \textbf{\textit{(i)}} Data Preparation and Augmentation Techniques; \textbf{\textit{(ii)}} Selecting and Adapting state-of-the-art LLM Models; \textbf{\textit{(iii)}} Evaluation Procedures. \texttt{SecRepair} uses a reinforcement learning-based fine-tuning with a semantic reward that caters to the functionality and security aspects of the generated code. Our empirical analysis shows that \texttt{SecRepair} achieves a \textit{12}\% improvement in security code repair compared to other LLMs when trained using reinforcement learning. Furthermore, we demonstrate the capabilities of \texttt{SecRepair} in generating reliable, functional, and compilable security code repairs against real-world test cases using automated evaluation metrics.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。