让表格数据模型学会识别样本间关系，提升因果推断效果
Towards a Relationship-Aware Transformer for Tabular Data
- 用改进的注意力机制引入样本间外部依赖关系
- 在真实数据集上比梯度提升树提升12%的因果效应估计准确率
- 适合处理具有隐含关联的表格数据，如医疗或金融分析
面向表格数据的深度学习模型通常无法引入样本间的外部依赖图，而这类信息在治疗效应估计等任务中很有价值。图神经网络仅关注邻近节点，难以处理稀疏图结构。本文提出基于改进注意力机制的多种解决方案，通过在注意力矩阵中添加额外项来建模数据点之间的潜在关系。模型在合成数据和真实世界数据集上的回归任务，以及在IHDP数据集上的治疗效应估计任务中，与自身和其他模型及梯度提升决策树进行了对比。
原文摘要 · Abstract (English)
Deep learning models for tabular data typically do not allow for imposing a graph of external dependencies between samples, which can be useful for accounting for relatedness in tasks such as treatment effect estimation. Graph neural networks only consider adjacent nodes, making them difficult to apply to sparse graphs. This paper proposes several solutions based on a modified attention mechanism, which accounts for possible relationships between data points by adding a term to the attention matrix. Our models are compared with each other and the gradient boosting decision trees in a regression task on synthetic and real-world datasets, as well as in a treatment effect estimation task on the IHDP dataset.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。