用结构化自然语言提升需求形式化一致性,让LLM推理更可信
Coherency through formalisations of Structured Natural Language, A case study on FRETish

- 以结构化自然语言为桥梁,确保多层需求描述逻辑一致
- 新翻译方法在MTL上验证等价性,统计显示效果更优
- 适合形式化验证、LLM推理安全研究者参考
形式化是将自然语言需求转化为形式语言的过程,常被视为验证中最复杂环节。现有工具在自然语言、技术语言、图示与形式语言间切换时,缺乏统一逻辑结构。本文提出「一致性形式化」新准则:各层次描述应保持相近逻辑结构。该原则对使用结构化自然语言作为中间层的LLM推理任务尤为重要。基于此,我们分析了NASA的FRET工具,提出将控制自然语言FRETish自动转换为MTL形式语言的新方法,并通过模型检测证明其与原译文等价。统计结果显示新译法更具优势,转化过程也揭示出原有规范中的不一致问题,值得深入讨论。
原文摘要 · Abstract (English)
Formalisation is the process of writing system requirements in a formal language. These requirements mostly originate in Natural Language. In the field of Formal Methods, formalisation is often identified as one of the most delicate and complicated steps in the verification process. Not seldomly, formalisation tools and environments choose various levels of requirement descriptions: Natural Language, Technical Language, Diagram Representations and Formal Language, to mention a few. In the literature, there are various maxims and principles of good practice to guide the process of requirement formalisation. In this paper we propose a new guideline: Coherency through Formalisations. The guideline states that the different levels of formalisation mentioned above should roughly follow the same logical structure. The principle seems particularly relevant in the setting where LLMs are prompted to perform reasoning tasks that can be checked by formal tools using Structured Natural Language to act as an intermediate layer bridging both paradigms. In the light of coherency, we analyze NASA's Formal Requirement Elicitation Tool FRET and propose an alternative automated translation of the Controlled Natural Language FRETish to the formal language of MTL. We compare our translation to the original translation and prove equivalence using model checking. Some statistics are performed which seem to favor the new translation. As expected, the translation process yielded interesting reflections and revealed inconsistencies which we present and discuss.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。