用知识蒸馏让小模型高效精准识别C/C++漏洞
Distilling Lightweight Language Models for C/C++ Vulnerabilities
- 用大模型教小模型，实现轻量级代码漏洞检测
- 在真实C/C++代码上比大模型更准且更快
- 适合需要高效部署的软件安全团队使用
现代软件系统日益复杂，安全漏洞频发，可能引发严重安全事故和巨额经济损失。因此，强大的代码漏洞检测能力对软件安全至关重要。尽管大型语言模型（LLMs）在自然语言处理中表现卓越，但其在自动化代码漏洞检测中的潜力尚未充分挖掘。本文提出FineSec框架，通过知识蒸馏技术，将大型教师模型的知识迁移至紧凑的学生模型，实现高效且精确的C/C++代码漏洞识别。FineSec将数据准备、训练、评估与持续学习整合为统一的单任务流程，显著简化了工作流。在多个真实C/C++代码库上的大量实验表明，FineSec在识别复杂漏洞和逻辑缺陷方面优于基础模型及更大规模的LLM，展现出良好的实用性和可扩展性。为确保可复现性，相关数据集、源码与实验结果已公开发布于：https://github.com/yangxiaoxuan123/FineSec_detect。
原文摘要 · Abstract (English)
The increasing complexity of modern software systems exacerbates the prevalence of security vulnerabilities, posing risks of severe breaches and substantial economic loss. Consequently, robust code vulnerability detection is essential for software security. While Large Language Models (LLMs) have demonstrated remarkable capabilities in natural language processing, their potential for automated code vulnerability detection remains underexplored. This paper presents FineSec, a novel framework that harnesses LLMs through knowledge distillation to enable efficient and precise vulnerability identification in C/C++ codebases. FineSec utilizes knowledge distillation to transfer expertise from large teacher models to compact student models, achieving high accuracy with minimal computational cost. By integrating data preparation, training, evaluation, and continuous learning into a unified, single-task workflow, FineSec offers a streamlined approach. Extensive evaluations on C/C++ codebases demonstrate its superiority over both base models and larger LLMs in identifying complex vulnerabilities and logical flaws, establishing FineSec as a practical and scalable solution for real-world software security. To facilitate reproducibility, the datasets, source code, and experimental results are made publicly available at: https://github.com/yangxiaoxuan123/FineSec_detect.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。