arXiv:2412.07437cs.LG2024-12被引 9

采样时机影响欺诈检测模型性能,提前采样会因数据泄露导致结果虚高。

Impact of Sampling Techniques and Data Leakage on XGBoost Performance in Credit Card Fraud Detection

  • 在训练集后分治时采样,避免数据泄露。
  • 预分治采样使准确率虚高,真实表现反而更差。
  • 适合关注评估可靠性与模型泛化能力的研究者。

信用卡欺诈检测是金融安全中的关键挑战,机器学习模型如XGBoost在识别欺诈交易方面表现强大。然而,信用卡交易数据普遍存在类别不平衡问题,严重影响模型性能。尽管采样技术常被用于缓解此问题,但若在划分训练/测试集前进行采样,可能引发数据泄露。本研究对比了三种场景下XGBoost在欺诈检测中的表现:不处理不平衡、采样仅在训练集后进行、采样在分治前进行。基于Kaggle提供的284,807条交易数据(其中0.172%为欺诈),结果显示:虽然采样能提升性能,但应用时机至关重要。在分治前采样会导致模型评估结果被人为夸大,而仅在训练集采样虽性能较低,却保证了评估的可靠性。

原文摘要 · Abstract (English)

Credit card fraud detection remains a critical challenge in financial security, with machine learning models like XGBoost(eXtreme gradient boosting) emerging as powerful tools for identifying fraudulent transactions. However, the inherent class imbalance in credit card transaction datasets poses significant challenges for model performance. Although sampling techniques are commonly used to address this imbalance, their implementation sometimes precedes the train-test split, potentially introducing data leakage. This study presents a comparative analysis of XGBoost's performance in credit card fraud detection under three scenarios: Firstly without any imbalance handling techniques, secondly with sampling techniques applied only to the training set after the train-test split, and third with sampling techniques applied before the train-test split. We utilized a dataset from Kaggle of 284,807 credit card transactions, containing 0.172\% fraudulent cases, to evaluate these approaches. Our findings show that although sampling strategies enhance model performance, the reliability of results is greatly impacted by when they are applied. Due to a data leakage issue that frequently occurs in machine learning models during the sampling phase, XGBoost models trained on data where sampling was applied prior to the train-test split may have displayed artificially inflated performance metrics. Surprisingly, models trained with sampling techniques applied solely to the training set demonstrated significantly lower results than those with pre-split sampling, all the while preserving the integrity of the evaluation process.

欺诈检测数据泄露采样策略XGBoost

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。