不补全缺失值,直接用Transformer处理带缺失的表格数据。
Imputation-free Learning of Tabular Data with Missing Values using Incremental Feature Partitions in Transformer
- 用动态特征分块和注意力掩码直接处理缺失数据
- 17个数据集上表现优于11种主流方法,最高提升12.3%
- 适合数据缺失多、怕补全失真的实际应用
带有不同缺失模式的表格数据通常需通过任意插补策略预处理。由插补模型生成的合成值常引发数据质量与数据驱动结果可靠性方面的担忧。为此,本文提出一种无需插补的增量注意力学习(IFIAL)方法,用于处理含缺失值的表格数据。通过推导并重构一对注意力掩码,使Transformer能直接处理原始表格数据,无需对缺失值进行插补或初始化。该方法通过增量学习重叠且固定大小的特征子集,提升Transformer性能。在17个多样化表格数据集上的平均分类性能排名表明,IFIAL优于11种现有先进方法(无论是否使用插补)。额外实验验证了IFIAL对不同缺失类型和比例的鲁棒性,其表现显著优于依赖显式插补的方法。当特征分块大小为原特征空间的一半时,计算效率与预测性能达到最佳平衡。IFIAL是首批使深度注意力模型可直接学习原始表格数据的方法之一,彻底免除插补需求。论文源码已公开。
原文摘要 · Abstract (English)
Tabular data sets with varying missing values are prepared for machine learning using an arbitrary imputation strategy. Synthetic values generated by imputation models often raise concerns regarding data quality and the reliability of data-driven outcomes. To address these concerns, this article proposes an imputation-free incremental attention learning (IFIAL) method for tabular data with missing values. A pair of attention masks is derived and retrofitted to a transformer to directly streamline tabular data without imputing or initializing missing values. The proposed method incrementally learns partitions of overlapping and fixed-size feature sets to enhance the performance of the transformer. The average classification performance rank order across 17 diverse tabular data sets highlights the superiority of IFIAL over 11 state-of-the-art learning methods with or without missing value imputations. Additional experiments corroborate the robustness of IFIAL to varying types and proportions of missing data, demonstrating its superiority over methods that rely on explicit imputations. A feature partition size equal to one-half the original feature space yields the best trade-off between computational efficiency and predictive performance. IFIAL is one of the first solutions that enables deep attention models to learn directly from tabular data, eliminating the need to impute missing values. %without the need for imputing missing values. The source code for this paper is publicly available.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。