用大模型多智能体自动生成仿真测试关系,提升工业仿真模型验证效率。
Multi-Agent Specification-based Metamorphic Testing of FMU-Based Simulations

- 基于规范构建多智能体系统,自动提取测试变换关系
- 在润滑油冷却系统上验证,可生成有效测试用例
- 适合需要自动化验证的复杂动态仿真系统开发者
在多个工业领域中,功能模拟接口(FMI)被用于在不同合作伙伴之间交换仿真模型,以功能模拟单元(FMU)的形式使用各种建模工具。这为利用FMU进行基于仿真的验证与确认提供了可能,以确保系统的可靠行为。然而,由于缺乏明确的预期输出,为这些仿真模型设计有效的测试断言仍然具有挑战性,这限制了传统测试方法的应用,后者通常需要访问系统的内部机制。元测试(MT)通过利用元关系(MRs)解决了这一局限,但从规范中提取此类关系仍主要是手动且易出错的过程。为此,我们提出了一种基于大语言模型(LLM)的多智能体工作流,用于基于规范的FMU仿真模型元测试。该方法以功能和接口规范为输入,协调多个智能体提取需求并推导出元关系。这些关系采用“给定-当-则”模式表达,结构化输入条件(给定)、变换操作(当)和预期输出行为(则)。随后,这些关系用于生成元测试用例,执行仿真,并在多次运行中评估输出一致性。我们在一个润滑油冷却系统FMU上进行了评估,证明该方法能够自动生成有意义的元关系及其对应的测试用例。初步结果表明,所提出的流程可通过减少人工识别元关系和生成测试用例的工作量,有效支持动态仿真模型的系统化验证与确认。
原文摘要 · Abstract (English)
In many industrial domains, the Functional Mock-up Interface (FMI) is used to exchange simulation models as Functional Mock-up Units (FMUs) across different partners using various modelling tools. This opens up the possibilities for simulation-based verification and validation using FMUs for ensuring reliable system behaviour. However, deriving effective test oracles for these simulation models remains challenging due to the absence of explicit expected outputs. This limits the applicability of conventional testing approaches, which require access to the internal workings of the systems. Metamorphic testing (MT) addresses this limitation by leveraging metamorphic relations (MRs), but extracting such relations from specifications remains largely a manual and error-prone process. To address this challenge, we propose an LLM-powered multi-agent workflow for specification-based metamorphic testing of FMU-based simulation models. The approach takes functional and interface specifications as input and orchestrates multiple agents to extract requirements and derive MRs. These MRs are expressed using Given-When-Then patterns to structure input conditions (Given), transformations (When), and expected output behaviours (Then). These relations are then used to generate metamorphic test cases, execute simulations, and evaluate output consistency across multiple sessions. We evaluate the approach on a Lube Oil Cooling system FMU, demonstrating its ability to automatically generate meaningful MRs and corresponding test cases. Preliminary results indicate that the proposed workflow can effectively support the systematic verification and validation of dynamic simulation models by reducing manual effort and improving test generation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。