用GPT-4o检测业务流程中的返工异常,准确率超96%
Leveraging GPT-4o Efficiency for Detecting Rework Anomaly in Business Processes
- 用GPT-4o将事件日志转为结构化数据并识别返工活动
- 在正常分布下用单样本提示达96.14%准确率
- 提示策略与异常分布显著影响检测效果,适合流程审计场景
本文研究OpenAI的大型语言模型GPT-4o-2024-08-06在检测业务流程异常(特别是返工异常)中的有效性。我们开发了一款基于GPT-4o的工具,可将事件日志转化为结构化格式,并识别其中的返工活动。分析基于一个包含返工异常但无循环的合成数据集进行。通过零样本、单样本和少样本三种提示技术,在正常、均匀和指数分布的异常情形下评估模型性能。结果表明,GPT-4o-2024-08-06表现优异:在正常分布下使用单样本提示时准确率达96.14%,均匀分布下使用少样本提示达97.94%,指数分布下使用少样本提示达74.21%。这些结果凸显了该模型在事件日志返工异常检测中的潜力,也揭示了异常分布与提示策略对模型性能的关键影响。
原文摘要 · Abstract (English)
This paper investigates the effectiveness of GPT-4o-2024-08-06, one of the Large Language Models (LLM) from OpenAI, in detecting business process anomalies, with a focus on rework anomalies. In our study, we developed a GPT-4o-based tool capable of transforming event logs into a structured format and identifying reworked activities within business event logs. The analysis was performed on a synthetic dataset designed to contain rework anomalies but free of loops. To evaluate the anomaly detection capabilities of GPT 4o-2024-08-06, we used three prompting techniques: zero-shot, one-shot, and few-shot. These techniques were tested on different anomaly distributions, namely normal, uniform, and exponential, to identify the most effective approach for each case. The results demonstrate the strong performance of GPT-4o-2024-08-06. On our dataset, the model achieved 96.14% accuracy with one-shot prompting for the normal distribution, 97.94% accuracy with few-shot prompting for the uniform distribution, and 74.21% accuracy with few-shot prompting for the exponential distribution. These results highlight the model's potential as a reliable tool for detecting rework anomalies in event logs and how anomaly distribution and prompting strategy influence the model's performance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。