用大模型自动监控代码运行时流程，发现异常更准更快。
Architecting software monitors for control-flow anomaly detection through large language models and conformance checking
- 用大模型自动连接设计模型与代码，生成执行日志。
- 在铁路系统测试中，异常检测F1最高达95.957%。
- 适合关注系统安全与可靠性的开发人员使用。
现代计算机系统因复杂性导致运行时行为可能偏离设计预期，产生难以预见的控制流异常。为提升系统可靠性，本文提出一种基于大语言模型（LLM）与一致性检查的软件监控方法，用于实时检测控制流异常。该方法通过LLM将设计阶段的模型与实现代码关联，自动完成源码埋点，生成事件日志；再利用可解释的一致性检查技术分析日志，识别偏离。在欧洲铁路交通管理系统/欧洲列车控制系统（ERTMS/ETCS）案例中，该方法实现高达82.849%的设计模型控制流覆盖率，后续一致性检查的异常检测性能达到95.957% F1-score和93.669% AUC。结果表明，引入领域知识引导大模型进行源码埋点，可生成高质量日志，有效支持控制流异常检测。
原文摘要 · Abstract (English)
Context: Ensuring high levels of dependability in modern computer-based systems has become increasingly challenging due to their complexity. Although systems are validated at design time, their behavior can be different at runtime, possibly showing control-flow anomalies due to ``unknown unknowns''. Objective: We aim to detect control-flow anomalies through software monitoring, which verifies runtime behavior by logging software execution and detecting deviations from expected control flow. Methods: We propose a methodology to develop software monitors for control-flow anomaly detection through Large Language Models (LLMs) and conformance checking. The methodology builds on existing software development practices to maintain traditional V\&V while providing an additional level of robustness and trustworthiness. It leverages LLMs to link design-time models and implementation code, automating source-code instrumentation. The resulting event logs are analyzed via conformance checking, an explainable and effective technique for control-flow anomaly detection. Results: We test the methodology on a case-study scenario from the European Railway Traffic Management System / European Train Control System (ERTMS/ETCS), which is a railway standard for modern interoperable railways. The results obtained from the ERTMS/ETCS case study demonstrate that LLM-based source-code instrumentation can achieve up to 82.849% control-flow coverage of the reference design-time process model, while the subsequent conformance checking-based anomaly detection reaches a peak performance of 95.957% F1-score and 93.669% AUC. Conclusion: Incorporating domain-specific knowledge to guide LLMs in source-code instrumentation significantly allowed obtaining reliable and quality software logs and enabled effective control-flow anomaly detection through conformance checking.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。