arXiv:2410.03399cs.LGcs.AI2024-10KDD被引 10

首个事件序列分类标准基准,解决模型评估无统一标准问题。

EBES: Easy Benchmarking for Event Sequences

  • 构建标准化评估流程与9种主流模型实现库
  • 涵盖10个真实与合成数据集,含最大公开银行数据集
  • 揭示事件序列独特性,指导模型选型与研究方向

事件序列(EvS)是具有不规则采样间隔及类别与数值特征混合的时序数据,其准确分类在医疗、金融和用户行为等领域至关重要。尽管该任务广受关注,但目前缺乏统一的基准与严谨的评估协议,导致研究结果难以比较,影响领域进展。为此,我们提出EBES——一个面向序列级目标的事件序列分类综合基准。EBES包含标准化评估场景与协议,开源的PyTorch实现库支持9种现代模型,并集成10个精选数据集,包括一个新型合成数据集和目前最大的公开银行数据集。库中提供便捷接口以整合新方法与数据集。基准测试揭示了事件序列相较于其他序列数据的独特属性,展示了模型性能排名:基于GRU的模型表现最佳,并识别出鲁棒学习的关键挑战。EBES旨在推动可复现研究,加速领域发展,提升技术实际应用价值。

原文摘要 · Abstract (English)

Event Sequences (EvS) refer to sequential data characterized by irregular sampling intervals and a mix of categorical and numerical features. Accurate classification of these sequences is crucial for various real-life applications, including healthcare, finance, and user interaction. Despite the popularity of the EvS classification task, there is currently no standardized benchmark or rigorous evaluation protocol. This lack of standardization makes it difficult to compare results across studies, which can result in unreliable conclusions and hinder progress in the field. To address this gap, we present EBES, a comprehensive benchmark for EvS classification with sequence-level targets. EBES features standardized evaluation scenarios and protocols, along with an open-source PyTorch library that implements 9 modern models. Additionally, it includes the largest collection of EvS datasets, featuring 10 curated datasets, including a novel synthetic dataset and real-world data with the largest publicly available banking dataset. The library offers user-friendly interfaces for integrating new methods and datasets. Our benchmarking results highlight the unique properties of EvS compared to other sequential data types, provide a performance ranking of modern models with GRU-based models achieving the best results and reveal the challenges associated with robust EvS learning. The goal of EBES is to facilitate reproducible research, expedite progress in the field, and increase the real-world impact of EvS classification techniques.

事件序列基准测试时序建模金融数据

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。