用检索增强生成技术检测流程异常,无需模型微调和专用流程模型。
Detecting Undesired Process Behavior by Means of Retrieval Augmented Generation
- 通过RAG让LLM直接访问其他流程的正反行为知识库。
- 在事件日志上下文增强下,检测准确率优于微调过的LLM。
- 适合缺乏流程模型且资源有限的工业场景使用。
合规性检查技术通过对比事件日志中记录的过程执行与专用流程模型中的预期行为,识别异常过程行为。当缺乏此类模型时,传统方法失效,但组织仍可能希望检测流程中的异常。现有方法依赖大型语言模型(LLMs),假设其可通过微调学习区分正常与异常行为,但微调成本高且泛化能力差。为此,本文提出一种新方法:无需专用流程模型,也无需资源密集型微调。该方法利用检索增强生成(RAG)使LLM直接访问包含其他流程正常与异常行为的知识库,假设模型能将知识迁移至当前流程。评估表明,该方法在检测异常行为方面优于微调后的LLMs,证明在结合事件日志中的高频轨迹和活动等上下文信息时,RAG是微调的有效替代方案。
原文摘要 · Abstract (English)
Conformance checking techniques detect undesired process behavior by comparing process executions that are recorded in event logs to desired behavior that is captured in a dedicated process model. If such models are not available, conformance checking techniques are not applicable, but organizations might still be interested in detecting undesired behavior in their processes. To enable this, existing approaches use Large Language Models (LLMs), assuming that they can learn to distinguish desired from undesired behavior through fine-tuning. However, fine-tuning is highly resource-intensive and the fine-tuned LLMs often do not generalize well. To address these limitations, we propose an approach that requires neither a dedicated process model nor resource-intensive fine-tuning to detect undesired process behavior. Instead, we use Retrieval Augmented Generation (RAG) to provide an LLM with direct access to a knowledge base that contains both desired and undesired process behavior from other processes, assuming that the LLM can transfer this knowledge to the process at hand. Our evaluation shows that our approach outperforms fine-tuned LLMs in detecting undesired behavior, demonstrating that RAG is a viable alternative to resource-intensive fine-tuning, particularly when enriched with relevant context from the event log, such as frequent traces and activities.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。