arXiv:2506.00943cs.SEcs.AI2025-06中稿 · publication at IEE…被引 2

用新指标评估大模型生成的智能合约法律合规性,发现大模型可作合规代码起点。

Legal Compliance Evaluation of Smart Contracts Generated By Large Language Models

  • 将法律与智能合约建模为流程,通过行为对比量化合规性。
  • 4个大模型生成20份合约,大模型表现更优,但需严格审查。
  • 新指标可细粒度比较代码合规性,适合自动化开发流程。

智能合约可实现并自动化部分法律合同内容,但确保其法律合规性仍具挑战。现有方法如形式化规范、验证和基于模型的开发,需同时具备法律与软件开发知识,且耗时费力。鉴于大语言模型在代码生成方面的进展,我们研究其直接从自然语言法律合同生成合法合规智能合约的能力。提出一套新指标,通过将法律与智能合约均建模为流程并比较其行为,量化法律合规性。选取4个大语言模型,基于5份法律合同生成20份智能合约,分析其法律合规性。结果表明,尽管所有模型生成的代码语法正确,但合规性存在显著差异,大模型普遍表现更佳。还评估了所提指标与软件度量属性的一致性,证明其能提供细粒度区分、支持细致比较,并适用于任何来源代码(大模型或开发者)。结果表明,大模型可用于生成合规性初步代码,结合严格审查;所提指标为自动化与自进化开发流程奠定基础。

原文摘要 · Abstract (English)

Smart contracts can implement and automate parts of legal contracts, but ensuring their legal compliance remains challenging. Existing approaches such as formal specification, verification, and model-based development require expertise in both legal and software development domains, as well as extensive manual effort. Given the recent advances of Large Language Models (LLMs) in code generation, we investigate their ability to generate legally compliant smart contracts directly from natural language legal contracts, addressing these challenges. We propose a novel suite of metrics to quantify legal compliance based on modeling both legal and smart contracts as processes and comparing their behaviors. We select four LLMs, generate 20 smart contracts based on five legal contracts, and analyze their legal compliance. We find that while all LLMs generate syntactically correct code, there is significant variance in their legal compliance with larger models generally showing higher levels of compliance. We also evaluate the proposed metrics against properties of software metrics, showing they provide fine-grained distinctions, enable nuanced comparisons, and are applicable across domains for code from any source, LLM or developer. Our results suggest that LLMs can assist in generating starter code for legally compliant smart contracts with strict reviews, and the proposed metrics provide a foundation for automated and self-improving development workflows.

智能合约大模型法律合规代码评估

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。