首个智能合约审计数据集，助力机器学习自动发现漏洞。
SC-Bench: A Large-Scale Dataset for Smart Contract Auditing
- 构建包含5377个真实合约与15975个违规的审计数据集
- 用GPT-4测试显示，有提示时仅22.9%漏洞可被检测
- 适合研究自动化合约安全检测的学者和开发者
为保障区块链平台上智能合约的安全与经济合规性，亟需高效审计手段。当前主要依赖人工审计，成本高且效率低。尽管机器学习有望降低人力负担，但尚无系统性方法应用于该领域。本文提出SC-Bench，首个面向自动化智能合约审计研究的大规模数据集。该数据集包含5,377个运行于以太坊平台的真实智能合约，以及15,975条以太坊标准（ERC）违规记录，其中139条为程序员真实失误，其余为系统注入的模拟违规。我们使用GPT-4对合约和规则进行测试，通过提供违规位置（即oracle）前后对比：无oracle时仅能检测0.9%的漏洞，有oracle时提升至22.9%。结果表明，现有ML技术在智能合约审计中仍有巨大改进空间。
原文摘要 · Abstract (English)
There is a huge demand to ensure the compliance of smart contracts listed on blockchain platforms to safety and economic standards. Today, manual efforts in the form of auditing are commonly used to achieve this goal. ML-based automated techniques have the promise to alleviate human efforts and the resulting monetary costs. However, unlike other domains where ML techniques have had huge successes, no systematic ML techniques have been proposed or applied to smart contract auditing. We present SC-Bench, the first dataset for automated smart-contract auditing research. SC-Bench consists of 5,377 real-world smart contracts running on Ethereum, a widely used blockchain platform, and 15,975 violations of standards on Ehereum called ERCs. Out of these violations, 139 are real violations programmers made. The remaining are errors we systematically injected to reflect the violations of different ERC rules. We evaluate SC-Bench using GPT-4 by prompting it with both the contracts and ERC rules. In addition, we manually identify each violated rule and the corresponding code site (i.e., oracle) and prompt GPT-4 with the information asking for a True-or-False question. Our results show that without the oracle, GPT-4 can only detect 0.9% violations, and with the oracle, it detects 22.9% violations. These results show the potential room for improvement in ML-based techniques for smart-contract auditing.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。