用GPT-2嵌入检测编译代码漏洞，准确率92.5%
Comparing Unidirectional, Bidirectional, and Word2vec Models for Discovering Vulnerabilities in Compiled Lifted Code
- 用GPT-2生成编译函数嵌入，替代BERT等双向模型
- 模型达92.5%准确率，F1-score 89.7%，优于双向模型
- 不冻结嵌入层+SGD优化器效果最佳，适合安全研究者
勒索软件等恶意程序通过长期存在的隐蔽漏洞造成重大损失。为检测编译代码中的缓冲区溢出等漏洞，本研究探索使用基于单向Transformer的GPT-2生成嵌入表示。基于LLVM函数数据集训练GPT-2模型生成嵌入，并用于构建LSTM神经网络以区分有无漏洞的代码。结果显示，GPT-2嵌入显著优于BERT和RoBERTa等双向模型，在测试集上达到92.5%准确率和89.7% F1-score。研究还发现，嵌入层未冻结时性能更优；在优化器对比中，SGD表现优于Adam。这些结果揭示了单向Transformer在提升网络安全防御方面的潜力。
原文摘要 · Abstract (English)
Ransomware and other forms of malware cause significant financial and operational damage to organizations by exploiting long-standing and often difficult-to-detect software vulnerabilities. To detect vulnerabilities such as buffer overflows in compiled code, this research investigates the application of unidirectional transformer-based embeddings, specifically GPT-2. Using a dataset of LLVM functions, we trained a GPT-2 model to generate embeddings, which were subsequently used to build LSTM neural networks to differentiate between vulnerable and non-vulnerable code. Our study reveals that embeddings from the GPT-2 model significantly outperform those from bidirectional models of BERT and RoBERTa, achieving an accuracy of 92.5% and an F1-score of 89.7%. LSTM neural networks were developed with both frozen and unfrozen embedding model layers. The model with the highest performance was achieved when the embedding layers were unfrozen. Further, the research finds that, in exploring the impact of different optimizers within this domain, the SGD optimizer demonstrates superior performance over Adam. Overall, these findings reveal important insights into the potential of unidirectional transformer-based approaches in enhancing cybersecurity defenses.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。