提出TEDD方法,自动检测时间数据中不稳定的特征。
TEDD: Robust Detection of Unstable Temporal Features

- 用回归模型预测时间戳,识别影响时间分布的特征。
- 可检测数值与类别型特征的各类变化模式,且无需调参。
- 适合数据科学家在模型部署前发现潜在漂移问题。
真实世界的时间数据常存在特征分布随时间变化的问题。若直接使用机器学习模型处理此类不稳定数据,性能可能迅速下降,尤其当新分布与训练数据差异显著时。为应对该问题,关键在于自动识别随时间变化的特征。本文提出TEDD方法,可判断数据集是否会导致模型不稳定,并自动定位引发不稳健性的具体特征。该方法利用回归模型预测实例的时间戳,从而识别对时间有强依赖的特征。在真实与合成数据上对比多种方法,验证其能检测所有基础变化类型(包括多变量漂移),对每项特征提供可比的改变程度度量,且无需参数调整,具备良好的可扩展性(支持大量特征和样本)。
原文摘要 · Abstract (English)
When working with real-world temporal data, it is common to encounter features whose distribution is changing over time. The naive employment of Machine Learning models on this unstable data might lead to rapidly degrading performance, especially if the new distribution is much different from what was previously seen during training. In order to cope with this problem, it is critical to automatically identify features that are changing over time. With these features detected, data scientists and other practitioners will be able to mitigate the issue (for instance, by applying data transformations), deploying more robust models that retain high performance for longer periods of time. In this paper, we describe which temporal changes a feature should not suffer from, and propose TEDD, a technique to a) identify when a dataset might lead to an unstable Machine Learning model and b) automatically detect which features cause such lack of robustness. In order to achieve it, we leverage a regression model to highlight which features contribute to a good prediction of an instance's timestamp. We compare our approach to other methods in real and synthetic data, testing their detection capability on all simple change patterns. We show that our method: detects all types of basic changes, both for numerical and categorical features; can detect multivariate drifts; returns a comparable value measuring the amount of change of each feature; requires no parameter tuning; and is scalable both on number of features and instances of the dataset.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。