一个管道生成多种专用表格缺失值填补模型,适应不同缺失模式。
One Pipeline, Many Transformers: Pattern-Specific Imputation Specialists for Tabular Missing Data
- 用Transformer构建基于行-列上下文的预测模型,统一处理缺失数据。
- 在42个数据集上验证,特定模式模型优于14种现有方法。
- 仅更换缺失机制模块即可生成专用模型,适合医疗金融等复杂场景。
表格数据中的缺失值迫使实践者在通用填补器(性能差)与等待专用算法之间二选一。现实中缺失常为非随机(MNAR),即缺失概率依赖于缺失值本身,违背经典完全随机缺失(MCAR)假设。本文提出单一预训练管道,通过三项设计实现:逐条特征化将填补问题转为行-列上下文监督预测;可插拔缺失机制的合成数据生成器;在百万张合成表上进行先验数据拟合。仅替换缺失机制模块,不改动架构、损失或训练流程,即可生成针对特定缺失模式的专家模型,且表现优于专为该模式设计的方法。我们在新基准MissBench上验证,涵盖42个OpenML数据集与11种缺失模式(含9种MNAR变体),覆盖医学、金融与工程领域。令人惊讶的是,仅在MCAR数据上训练的默认模型(TabImpute)在所有测试模式下均表现稳健。此外,本管道生成的模式专用模型在各自目标模式上超越14个既有基线。代码、模型与基准已开源。
原文摘要 · Abstract (English)
Missing data in tabular datasets forces practitioners into a hard choice: deploy a general-purpose imputer that may perform poorly for the problem at hand, or wait for someone to design a specialized algorithm. This problem is worsened by the fact that real-world missingness rarely satisfies the textbook missing completely at random (MCAR) assumption, as entries are often missing not at random (MNAR), where the probability of being observed depends on the missing data itself. We collapse this trade-off into a single pre-training pipeline that builds transformer-based imputation specialists through three components: an entry-wise featurization that recasts imputation as supervised prediction over row--column context, a synthetic data generator with pluggable missingness modules, and prior-data fitting on millions of synthetic tables. Swapping only the missingness module, with no changes to architecture, loss, or training, yields a pattern-specific specialist that outperforms methods purpose-built for that pattern. We validate this on MissBench, a new benchmark of 42 OpenML datasets and 11 missingness patterns (including 9 MNAR variants) spanning medicine, finance, and engineering. Remarkably, training exclusively on MCAR yields a default model---TabImpute---robust across all tested patterns. In addition, the pattern-specific specialists produced by our pipeline outperform the 14 established baselines on their target patterns. We open-source the pipeline, models, and benchmark.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。