重新评估时间序列分类基准,发现多数数据集无需依赖时序信息
Revisit Time Series Classification Benchmark: The Impact of Temporal Information for Classification
- 通过打乱时序的置换测试,检验时间序列数据的真实依赖性
- 超半数UCR数据集分类性能不受时序破坏影响,暴露评估偏差
- 提出UCR Augmented新基准,更公平衡量模型时序特征提取能力
时间序列分类通常被认为与表格数据分类不同,因其依赖时序信息。然而,本文通过对最常用的时间序列分类基准UCR时间序列分类档案进行置换测试(打乱时间顺序),发现其中相当一部分数据集的时间信息对分类几乎无影响。许多数据集本质上是表格型或主要依赖非时序特征,导致以时序为核心的设计在评估中产生偏差。为此,我们提出了基于UCR档案的UCR Augmented新基准,用于评估分类器提取和利用时序信息的能力。在该基准上测试来自七个类别共17种分类器,性能排名出现显著变化:部分此前被忽视的方法表现优异,而一些原优势方法在时序关键任务中大幅退步。该基准为时间序列分类器提供了更稳健、更公平的评估框架。代码已公开于https://github.com/YunruiZhang/Revisit-Time-Series-Classification-Benchmark。
原文摘要 · Abstract (English)
Time series classification is usually regarded as a distinct task from tabular data classification due to the importance of temporal information. However, in this paper, by performing permutation tests that disrupt temporal information on the UCR time series classification archive, the most widely used benchmark for time series classification, we identify a significant proportion of datasets where temporal information has little to no impact on classification. Many of these datasets are tabular in nature or rely mainly on tabular features, leading to potentially biased evaluations of time series classifiers focused on temporal information. To address this, we propose UCR Augmented, a benchmark based on the UCR time series classification archive designed to evaluate classifiers' ability to extract and utilize temporal information. Testing classifiers from seven categories on this benchmark revealed notable shifts in performance rankings. Some previously overlooked approaches perform well, while others see their performance decline significantly when temporal information is crucial. UCR Augmented provides a more robust framework for assessing time series classifiers, ensuring fairer evaluations. Our code is available at https://github.com/YunruiZhang/Revisit-Time-Series-Classification-Benchmark.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。