针对表格数据缺失分布不均问题，提出按比例掩码的自编码器方法。
To Predict or Not To Predict? Proportionally Masked Autoencoders for Tabular Data Imputation
- 按数据实际缺失比例生成掩码，保持缺失分布一致性。
- 用MLP替代注意力机制，效率更高且性能相当。
- 适用于各类缺失模式的表格数据修复，尤其适合异构数据。
掩码自编码器（MAEs）在表格数据填补中表现良好。然而，由于表格数据固有的异质性，当前普遍采用的随机均匀掩码策略会破坏缺失分布，导致性能不佳。为此，我们提出一种比例掩码策略：首先根据数据集中观测到的缺失比例计算统计量，再生成与之匹配的掩码，确保掩码后缺失分布保持不变。此外，我们指出，在表格数据场景下，简单的MLP型令牌混合相比注意力机制具有竞争力甚至更优的性能，同时计算效率更高。实验结果表明，所提比例掩码策略在多种缺失模式的表格数据上均有效。代码已开源：https://github.com/normal-kim/PMAE。
原文摘要 · Abstract (English)
Masked autoencoders (MAEs) have recently demonstrated effectiveness in tabular data imputation. However, due to the inherent heterogeneity of tabular data, the uniform random masking strategy commonly used in MAEs can disrupt the distribution of missingness, leading to suboptimal performance. To address this, we propose a proportional masking strategy for MAEs. Specifically, we first compute the statistics of missingness based on the observed proportions in the dataset, and then generate masks that align with these statistics, ensuring that the distribution of missingness is preserved after masking. Furthermore, we argue that simple MLP-based token mixing offers competitive or often superior performance compared to attention mechanisms while being more computationally efficient, especially in the tabular domain with the inherent heterogeneity. Experimental results validate the effectiveness of the proposed proportional masking strategy across various missing data patterns in tabular datasets. Code is available at: \url{https://github.com/normal-kim/PMAE}.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。