简单计数法竟在预测下一步活动时胜过大模型。
David vs. Goliath in Next Activity Prediction: Argmax vs. LSTM, Transformer, and LLM
- 用真实数据集对比大模型与简单方法在序列预测中的表现。
- 多数数据集上，计数法性能接近甚至超过千亿参数大模型。
- 模型大小和预训练对结果影响不大，适合快速部署场景。
下一步活动预测（NAP）是预测性流程监控（PPM）的核心，使组织能从回顾分析转向主动流程调控。该领域已从传统机器学习发展到LSTM、Transformer，再到大语言模型（LLMs）。尽管模型日益复杂，但尚无直接比较LLMs、Transformer、LSTM与简单基线的统一基准。本文填补此空白，系统性地评估了适配词表的LLMs、从头训练的Transformer、LLM蒸馏的Transformer以及LSTM，并与基于计数的argmax基线在七个真实事件日志上进行对比。结果显示：预训练并未带来稳定提升，模型规模对性能影响有限，在多数数据集上，argmax基线的表现与千亿参数LLM相当甚至更优。
原文摘要 · Abstract (English)
Next activity prediction (NAP) is a cornerstone of predictive process monitoring (PPM), enabling organizations to move from retrospective analysis to proactive process steering. The PPM field has progressed from classical machine learning through deep learning architectures such as LSTMs and Transformers to large language models (LLMs). Despite growing model complexity, no benchmark jointly compares LLMs, Transformers, LSTMs, and simple baselines in a direct sequence modeling setting for NAP. In this paper, we fill this gap with a systematic benchmark. We compare vocabulary-adapted LLMs, Transformers trained from scratch, LLM-distilled Transformers, and LSTMs against a simple counting-based argmax baseline across seven real-life event logs. Our results tell a David vs. Goliath story: pretraining confers no consistent improvement over training from scratch, model size shows little effect on performance, and on most datasets the argmax baseline matches or approaches the performance of billion-parameter LLMs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。