新方法让表格数据模型自适应时间分布变化,秒级预测且无需调参。
Drift-Resilient TabPFN: In-Context Learning Temporal Distribution Shifts on Tabular Data

- 用先验因果模型学习元学习算法,实现上下文内推理
- 在18个数据集上准确率提升至0.744,AUC达0.832
- 适合处理时间漂移的表格数据,尤其适合无调参场景
大多数机器学习模型假设数据独立同分布,但现实中的分布漂移常导致性能下降。目前尚无表格数据方法能持续超越忽略漂移的经典监督学习。为此,我们提出Drift-Resilient TabPFN,一种基于先验数据拟合网络的上下文学习方法,可将整个训练集作为输入,在单次前向传播中完成测试集预测。该方法通过合成数据逼近贝叶斯推断,其先验基于随时间变化的结构因果模型(SCM),并通过二级SCM建模主模型参数变化。实验在18个合成与真实数据集上验证,相比XGB、CatBoost、TabPFN等基线,准确率从0.688提升至0.744,ROC AUC从0.786升至0.832,且校准性更强。该方法适用于未见数据,小到中等规模数据集可在秒级完成推理,无需超参数调优,为分布外预测研究奠定基础。
原文摘要 · Abstract (English)
While most ML models expect independent and identically distributed data, this assumption is often violated in real-world scenarios due to distribution shifts, resulting in the degradation of machine learning model performance. Until now, no tabular method has consistently outperformed classical supervised learning, which ignores these shifts. To address temporal distribution shifts, we present Drift-Resilient TabPFN, a fresh approach based on In-Context Learning with a Prior-Data Fitted Network that learns the learning algorithm itself: it accepts the entire training dataset as input and makes predictions on the test set in a single forward pass. Specifically, it learns to approximate Bayesian inference on synthetic datasets drawn from a prior that specifies the model's inductive bias. This prior is based on structural causal models (SCM), which gradually shift over time. To model shifts of these causal models, we use a secondary SCM, that specifies changes in the primary model parameters. The resulting Drift-Resilient TabPFN can be applied to unseen data, runs in seconds on small to moderately sized datasets and needs no hyperparameter tuning. Comprehensive evaluations across 18 synthetic and real-world datasets demonstrate large performance improvements over a wide range of baselines, such as XGB, CatBoost, TabPFN, and applicable methods featured in the Wild-Time benchmark. Compared to the strongest baselines, it improves accuracy from 0.688 to 0.744 and ROC AUC from 0.786 to 0.832 while maintaining stronger calibration. This approach could serve as significant groundwork for further research on out-of-distribution prediction.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。