实时检测金融欺诈,自动应对数据模式变化
ROSFD: Robust Online Streaming Fraud Detection with Resilience to Concept Drift in Data Streams
- 分两阶段:先离线初始化模型,再实时动态调整
- 用ADWIN检测数据漂移,减少重训练次数,保持高精度
- 适合需要持续监控的金融、电商反欺诈系统
来自在线交易和数字交互等多元来源的持续数据流,要求及时进行欺诈检测。传统批量处理方法难以捕捉欺诈行为的快速演变模式。本文强调实时处理数据流对有效反欺诈的重要性。为应对延迟、可扩展性和概念漂移等挑战,提出一种鲁棒的在线流式欺诈检测框架(ROSFD)。该框架包含两个关键阶段:(i) 离线模型初始化阶段,采用增量学习构建初始模型,解决冷启动问题;(ii) 实时模型自适应阶段,使用漂移检测算法(如DDM、EDDM、ADWIN)识别数据流中的概念漂移,并相应地增量训练模型。这种“按需重训练”策略显著减少了重训练次数,同时对受试者工作特征曲线下面积(AUC)影响较小。总体而言,采用ADWIN作为漂移检测器的ROSFD表现最佳;在模型性能方面,自适应随机森林在五个数据集中的四个上取得最高AUC。
原文摘要 · Abstract (English)
Continuous generation of streaming data from diverse sources, such as online transactions and digital interactions, necessitates timely fraud detection. Traditional batch processing methods often struggle to capture the rapidly evolving patterns of fraudulent activities. This paper highlights the critical importance of processing streaming data for effective fraud detection. To address the inherent challenges of latency, scalability, and concept drift in streaming environments, we propose a robust online streaming fraud detection (ROSFD) framework. Our proposed framework comprises two key stages: (i) Stage One: Offline Model Initialization. In this initial stage, a model is built in offline settings using incremental learning principles to overcome the "cold-start" problem. (ii) Stage Two: Real-time Model Adaptation. In this dynamic stage, drift detection algorithms (viz.,, DDM, EDDM, and ADWIN) are employed to identify concept drift in the incoming data stream and incrementally train the model accordingly. This "train-only-when-required" strategy drastically reduces the number of retrains needed without significantly impacting the area under the receiver operating characteristic curve (AUC). Overall, ROSFD utilizing ADWIN as the drift detection method demonstrated the best performance among the employed methods. In terms of model efficacy, Adaptive Random Forest consistently outperformed other models, achieving the highest AUC in four out of five datasets.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。