arXiv:2606.30441cs.MAcs.AI2026-06

用大模型把自然语言需求转为精确的多智能体逻辑公式,让非专家也能快速写对系统规范。

Translating Natural Language to Strategic Temporal Specifications via LLMs

论文配图:Translating Natural Language to Strategic Temporal Specifications via LLMs
图 1 · 摘自论文原文
  • 基于大模型将自然语言需求转化为标准的ATL/ATL*逻辑公式
  • 小模型微调后达0.84语义准确率,媲美闭源大模型
  • 支持本地部署,适合非专家在实际系统中使用

严谨的形式化系统需求是多智能体系统(MAS)验证的基础,但编写正确的形式化规范既易出错、耗时又需专业知识。这一挑战在多智能体系统中尤为突出,因需求必须涵盖战略能力与时间目标。目前尚无从自然语言生成MAS规范的成熟方法。本文提出一个利用大语言模型(LLMs)将自然语言描述的战略性要求转换为符合规范的ATL/ATL*公式的框架。由于缺乏可用于监督学习的标注数据集,我们构建并校验了一个全新的专家标注数据集,用于训练和评估微调模型。在保留测试集上,经由最贴近专家判断的LLM评判器评估,小型开源模型(3-7B参数)的微调表现与强少数样本闭源API基线相当。最佳微调系统达到0.84的语义准确率,与最强闭源基线的0.86统计上无显著差异,且可本地部署。我们还发现评判器可靠性与生成器强度呈负相关:开放权重的Llama-3.3-70B最接近人类判断,而最强闭源模型则最不可靠,过度拒绝忠实改写版本。为验证实用性,我们将工具嵌入现有战略逻辑模型检测器,使非专家用户可通过自然语言指定战略属性。

原文摘要 · Abstract (English)

A rigorous formalization of system requirements is a fundamental prerequisite for the verification of Multi-Agent Systems (MAS). However, writing correct formal specifications is well known as an error-prone, time-consuming, and expertise-intensive task. This difficulty is further accentuated in MAS, where requirements must capture strategic abilities and temporal objectives. At present, there is no established methodology for deriving MAS specifications from natural language. We present a framework for translating Natural Language descriptions of strategic requirements into well-formed ATL/ATL* formulas using Large Language Models (LLMs). Since no available dataset supports supervised learning for the NL-to-ATL/ATL* translation task, we create and curate a novel expert-validated dataset, employed for training and evaluating fine-tuned models. On a held-out test set, evaluated under the LLM judge that best agrees with expert annotations, in-domain fine-tuning of small open-weight models (3 - 7B parameters) matches strong few-shot proprietary API baselines. Our best fine-tuned system reaches 0.84 semantic accuracy, statistically on par with 0.86 for the strongest few-shot proprietary baseline, while keeping requirements on-premises. We further find that judge reliability is inverse to generator strength. The open-weight Llama-3.3-70B tracks human verdicts most closely, whereas the strongest proprietary models are the least reliable judges, over-rejecting faithful paraphrases of the reference. To assess the practical applicability of the generated specifications, we embed our tool to an existing strategic logics model checker, enabling non-expert users to specify strategic properties in natural language.

自然语言形式化验证多智能体大模型应用

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。