arXiv:2508.06337stat.MLcs.LG2025-08被引 1

解决特征相关时重要性评估失真的问题,让模型更准确判断哪些特征真正关键。

Decorrelated feature importance from local sample weighting

  • 通过局部样本加权消除特征间相关性,提升重要性评估的准确性
  • 在多种相关模式下均显著改善特征重要性得分,且对分布外数据预测精度提升
  • 可适配决策树与神经网络,有自然调参机制,适合需要可解释性的场景

特征重要性(FI)是理解机器学习模型决策过程的重要工具,但在特征存在相关性时其有效性受限。此时,重要性常被分散到所有与响应信号相关的特征上;若多个信号特征与噪声特征强相关但彼此弱相关,可能导致噪声特征获得更高的FI值。本文提出局部样本加权(losaw),可灵活集成于多种机器学习算法中,以改善特征相关情况下的FI评分。该方法受因果推断中逆概率加权启发,在模型内部局部使用样本加权,使目标特征与其他特征解耦,降低局部模型偏差。losaw包含一个自然调参项——加权后群体的最小有效样本量,对应解释-预测权衡,类似传统调参中的偏差-方差权衡。我们展示了losaw在基于决策树的方法及神经网络小批量训练中的集成方式,并在随机森林与卷积神经网络上,针对多样相关模式的模拟研究中验证了其效果:losaw能一致提升特征重要性,同时提升分布外数据的预测精度,而对分布内测试数据精度保持相近。

原文摘要 · Abstract (English)

Feature importance (FI) statistics provide a prominent and valuable method of insight into the decision process of machine learning (ML) models, but their effectiveness has well-known limitations when correlation is present among the features in the training data. In this case, the FI often tends to be distributed among all features which are in correlation with the response-generating signal features. Even worse, if multiple signal features are in strong correlation with a noise feature, while being only modestly correlated with one another, this can result in a noise feature having a distinctly larger FI score than any signal feature. Here we propose local sample weighting (losaw) which can flexibly be integrated into many ML algorithms to improve FI scores in the presence of feature correlation in the training data. Our approach is motivated from inverse probability weighting in causal inference and locally, within the ML model, uses a sample weighting scheme to decorrelate a target feature from the remaining features. This reduces model bias locally, whenever the effect of a potential signal feature is evaluated and compared to others. Moreover, losaw comes with a natural tuning parameter, the minimum effective sample size of the weighted population, which corresponds to an interpretation-prediction-tradeoff, analog to a bias-variance-tradeoff as for classical ML tuning parameters. We demonstrate how losaw can be integrated within decision tree-based ML methods and within mini-batch training of neural networks. We investigate losaw for random forest and convolutional neural networks in a simulation study on settings showing diverse correlation patterns. We found that losaw improves FI consistently. Moreover, it often improves prediction accuracy for out-of-distribution, while maintaining a similar accuracy for in-distribution test data.

特征重要性可解释性相关性处理机器学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。