用大模型检测智能合约漏洞，显著降低误报率。
Logic Meets Magic: LLMs Cracking Smart Contract Vulnerabilities
- 设计优化提示词，使误报率下降超60%。
- 新版本Solidity漏洞检出率仅13%，明显下降。
- 适合区块链安全研究者与开发人员参考。
智能合约漏洞导致区块链应用出现重大经济损失。大语言模型（LLMs）为解决这一耗时任务提供了新可能，但现有基于LLM的检测方法常面临高误报率问题。本文在两方面推进现有研究：首先，评估基于最新Solidity v0.8版本，提供比以往聚焦v0.4版本更前沿的见解；其次，采用五大领先LLM模型（跨公司），覆盖领域内最先进的能力。通过一系列严格实验发现，优化提示词可使误报率降低超过60%。令人意外的是，针对Solidity v0.8中某些特定漏洞的召回率已降至13%，相比v0.4版本大幅下降。深入分析揭示根本原因：LLMs依赖于识别新引入库和框架的变化来进行漏洞检测。
原文摘要 · Abstract (English)
Smart contract vulnerabilities caused significant economic losses in blockchain applications. Large Language Models (LLMs) provide new possibilities for addressing this time-consuming task. However, state-of-the-art LLM-based detection solutions are often plagued by high false-positive rates. In this paper, we push the boundaries of existing research in two key ways. First, our evaluation is based on Solidity v0.8, offering the most up-to-date insights compared to prior studies that focus on older versions (v0.4). Second, we leverage the latest five LLM models (across companies), ensuring comprehensive coverage across the most advanced capabilities in the field. We conducted a series of rigorous evaluations. Our experiments demonstrate that a well-designed prompt can reduce the false-positive rate by over 60%. Surprisingly, we also discovered that the recall rate for detecting some specific vulnerabilities in Solidity v0.8 has dropped to just 13% compared to earlier versions (i.e., v0.4). Further analysis reveals the root cause of this decline: the reliance of LLMs on identifying changes in newly introduced libraries and frameworks during detection.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。