arXiv:2510.12702cs.SEcs.AI2025-10被引 6

用大模型把代码注释转为完整规范,减少自动验证误报。

Beyond Postconditions: Can Large Language Models infer Formal Contracts for Automatic Software Verification?

  • 将自然语言注释转化为包含前提和后置条件的正式契约
  • 生成的契约能有效区分正确与错误代码行为
  • 相比仅依赖后置条件,大幅降低验证误报率

自动软件验证器在检查代码是否符合(形式化)规范方面越来越高效,但实际应用受限于真实代码中缺乏规范。大型语言模型(LLMs)已展现出从函数名、注释或文档等自然语言提示中推断形式化后置条件的能力。然而,仅使用生成的后置条件进行验证时,常导致验证器提出无效输入,引发大量误报。为此,本文重新审视了自然语言到形式规范的推断问题,提出NL2Contract任务:利用大模型将非正式自然语言转化为包含前置条件和后置条件的正式功能契约。我们设计了以保真性、缺陷区分能力及在自动验证中的可用性为核心指标的评估体系。实验表明,(1)大模型普遍能生成对所有输入都保真的契约;(2)生成契约具备足够表达力以区分有缺陷与正确行为;(3)使用大模型生成的完整契约可显著减少验证误报。进一步分析显示,生成的前置条件与开发者意图高度一致,使自动验证器能够发现真实世界中的缺陷。

原文摘要 · Abstract (English)

Automatic software verifiers have become increasingly effective at the task of checking software against (formal) specifications. Yet, their adoption in practice has been hampered by the lack of such specifications in real world code. Large Language Models (LLMs) have shown promise in inferring formal postconditions from natural language hints embedded in code such as function names, comments or documentation. Using the generated postconditions as specifications in a subsequent verification, however, often leads verifiers to suggest invalid inputs, hinting at potential issues that ultimately turn out to be false alarms. To address this, we revisit the problem of specification inference from natural language in the context of automatic software verification. In the process, we introduce NL2Contract, the task of employing LLMs to translate informal natural language into formal functional contracts, consisting of postconditions as well as preconditions. We introduce metrics to validate and compare different NL2Contract approaches, using soundness, bug discriminative power of the generated contracts and their usability in the context of automatic software verification as key metrics. We evaluate NL2Contract with different LLMs and compare it to the task of postcondition generation nl2postcond. Our evaluation shows that (1) LLMs are generally effective at generating functional contracts sound for all possible inputs, (2) the generated contracts are sufficiently expressive for discriminating buggy from correct behavior, and (3) verifiers supplied with LLM inferred functional contracts produce fewer false alarms than when provided with postconditions alone. Further investigations show that LLM inferred preconditions generally align well with developers intentions which allows us to use automatic software verifiers to catch real-world bugs.

形式化验证大模型代码生成契约推理

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。