发现时间序列归因评估中存在类别依赖偏差,影响结果可靠性。
Class-Dependent Perturbation Effects in Evaluating Time Series Attributions
- 通过系统实验揭示归因评估指标在不同类别上表现不一
- 最有效的扰动策略反而类别差异最明显,最高达37%
- 提出带类别感知惩罚的评估框架,适合不平衡数据场景
随着机器学习在时间序列应用中的普及,可解释人工智能(XAI)方法对于理解模型预测至关重要。特征归因方法旨在识别对模型预测贡献最大的输入特征,其评估通常依赖于基于扰动的指标。通过对多个数据集、模型架构和扰动策略进行系统性实证分析,我们揭示了以往被忽视的类别依赖效应:这些指标在不同类别上的有效性存在差异,对某些类别表现优异,而对其他类别则不敏感。特别地,最有效的扰动策略往往表现出最显著的类别差异。我们的分析表明,这些效应源于分类器的隐含偏置,提示基于扰动的评估可能反映特定模型行为而非内在归因质量。为此,我们提出一种包含类别感知惩罚项的评估框架,有助于在评估特征归因时识别并校正此类效应,尤其适用于类别不平衡的数据集。尽管本研究聚焦时间序列分类,这类类别依赖效应很可能也存在于其他使用基于扰动评估的结构化数据领域。
原文摘要 · Abstract (English)
As machine learning models become increasingly prevalent in time series applications, Explainable Artificial Intelligence (XAI) methods are essential for understanding their predictions. Within XAI, feature attribution methods aim to identify which input features contribute the most to a model's prediction, with their evaluation typically relying on perturbation-based metrics. Through systematic empirical analysis across multiple datasets, model architectures, and perturbation strategies, we reveal previously overlooked class-dependent effects in these metrics: they show varying effectiveness across classes, achieving strong results for some while remaining less sensitive to others. In particular, we find that the most effective perturbation strategies often demonstrate the most pronounced class differences. Our analysis suggests that these effects arise from the learned biases of classifiers, indicating that perturbation-based evaluation may reflect specific model behaviors rather than intrinsic attribution quality. We propose an evaluation framework with a class-aware penalty term to help assess and account for these effects in evaluating feature attributions, offering particular value for class-imbalanced datasets. Although our analysis focuses on time series classification, these class-dependent effects likely extend to other structured data domains where perturbation-based evaluation is common.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。