arXiv:2504.08712cs.LGstat.ML2025-04被引 3

让表格型Transformer模型既准确又可解释,识别每个特征的独立影响。

Beyond Black-Box Predictions: Identifying Marginal Feature Effects in Tabular Transformer Networks

  • 在Transformer中引入可解释性设计,保留特征的边际效应。
  • 在复杂交互下仍能准确检测特征影响,性能媲美黑盒模型。
  • 适合需要高精度且要求透明决策的工业场景使用。

近年来,深度神经网络在各类任务中展现出强大的预测能力。除自然语言处理外,Transformer架构在表格数据任务中也表现出色,挑战了传统梯度提升决策树的主导地位。然而,这种高性能伴随着可解释性的丧失:深表格式的Transformer几乎完全丢失了特征的边际效应。传统统计回归模型虽能保持可解释性,但预测性能通常逊于复杂模型。为此,我们提出一种改进的表格Transformer架构,旨在识别边际特征效应。理论证明该方法可准确识别边际效应,消融实验表明其在复杂特征交互下仍能有效检测。与多种可解释及黑盒模型对比,该模型在保持可解释性的同时,性能可与黑盒模型相当。代码已开源:https://github.com/OpenTabular/NAMpy。

原文摘要 · Abstract (English)

In recent years, deep neural networks have showcased their predictive power across a variety of tasks. Beyond natural language processing, the transformer architecture has proven efficient in addressing tabular data problems and challenges the previously dominant gradient-based decision trees in these areas. However, this predictive power comes at the cost of intelligibility: Marginal feature effects are almost completely lost in the black-box nature of deep tabular transformer networks. Alternative architectures that use the additivity constraints of classical statistical regression models can maintain intelligible marginal feature effects, but often fall short in predictive power compared to their more complex counterparts. To bridge the gap between intelligibility and performance, we propose an adaptation of tabular transformer networks designed to identify marginal feature effects. We provide theoretical justifications that marginal feature effects can be accurately identified, and our ablation study demonstrates that the proposed model efficiently detects these effects, even amidst complex feature interactions. To demonstrate the model's predictive capabilities, we compare it to several interpretable as well as black-box models and find that it can match black-box performances while maintaining intelligibility. The source code is available at https://github.com/OpenTabular/NAMpy.

可解释性表格数据Transformer

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。