首个针对反编译二进制漏洞分析的LLM数据集，显著提升模型检测能力。
Enhancing Reverse Engineering: Investigating and Benchmarking Large Language Models for Vulnerability Analysis in Decompiled Binaries
- 构建多架构、多优化级别的二进制漏洞数据集DeBinVul，覆盖15万+样本
- 在漏洞检测任务中，代码模型性能提升最高达24%
- 适用于安全研究者、逆向工程师及二进制漏洞自动化分析开发者
安全专家通过反编译二进制代码识别关键安全漏洞。由于关键基础设施（CI）中的固件、驱动和专有软件缺乏源码，二进制层面的分析尤为关键。即使有源码，编译后仍存在语义鸿沟，可能阻碍漏洞发现。当前大语言模型（LLMs）研究主要聚焦源码，忽视了反编译二进制的重要性。本文首次实证揭示主流LLMs在分析反编译二进制漏洞时存在显著语义局限，主因是缺乏相关数据集。为此，我们提出DeBinVul——一个面向C/C++的多架构、多优化级别二进制漏洞数据集，包含150,872个样本，支持漏洞识别、分类、描述与函数名恢复任务。基于该数据集微调CodeLlama、Llama3和CodeGen2，漏洞检测性能分别提升19%、24%和21%；漏洞分类准确率达80%-90%；函数名恢复与漏洞描述任务亦获明显改善。
原文摘要 · Abstract (English)
Security experts reverse engineer (decompile) binary code to identify critical security vulnerabilities. The limited access to source code in vital systems - such as firmware, drivers, and proprietary software used in Critical Infrastructures (CI) - makes this analysis even more crucial on the binary level. Even with available source code, a semantic gap persists after compilation between the source and the binary code executed by the processor. This gap may hinder the detection of vulnerabilities in source code. That being said, current research on Large Language Models (LLMs) overlooks the significance of decompiled binaries in this area by focusing solely on source code. In this work, we are the first to empirically uncover the substantial semantic limitations of state-of-the-art LLMs when it comes to analyzing vulnerabilities in decompiled binaries, largely due to the absence of relevant datasets. To bridge the gap, we introduce DeBinVul, a novel decompiled binary code vulnerability dataset. Our dataset is multi-architecture and multi-optimization, focusing on C/C++ due to their wide usage in CI and association with numerous vulnerabilities. Specifically, we curate 150,872 samples of vulnerable and non-vulnerable decompiled binary code for the task of (i) identifying; (ii) classifying; (iii) describing vulnerabilities; and (iv) recovering function names in the domain of decompiled binaries. Subsequently, we fine-tune state-of-the-art LLMs using DeBinVul and report on a performance increase of 19%, 24%, and 21% in the capabilities of CodeLlama, Llama3, and CodeGen2 respectively, in detecting binary code vulnerabilities. Additionally, using DeBinVul, we report a high performance of 80-90% on the vulnerability classification task. Furthermore, we report improved performance in function name recovery and vulnerability description tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。