arXiv:2607.26000cs.LGcs.AI2026-07

评估9种表格基础模型在分布外数据下的表现,发现均显著退化。

Empirical Evaluation of Out-Of-Distribution Performance of Tabular Foundation Models

  • 系统测试9种表格模型在真实分布偏移下的性能
  • 所有模型在分布外均退化,差距达0.003至0.060
  • 揭示高资源消耗与实际部署间的矛盾,适合高风险场景研究

表格基础模型(TFMs)在表格预测任务中展现出与集成树模型相当的性能。然而,多数TFMs在独立同分布数据上训练和评估,而现实场景中存在分布偏移,影响模型鲁棒性。目前对TFMs在分布偏移下的研究有限。本文对9种不同预训练策略与架构的TFMs进行了实证评估:TabPFNv2、TabPFNv2.5、TabPFNv2.6、TabPFNv3、TabICL、TabICLv2、Mitra、LimiX和TabFM。基于TableShift研究中的三个真实数据集(HELOC、Voting、Childhood Lead),覆盖标签、社会经济和地理偏移类型。结果表明,所有被评估的TFMs在分布外均出现系统性性能下降,偏差范围为0.003至0.060,取决于偏移类型。经典表格模型中观测到的分布内与分布外性能关系同样适用于TFMs。此外,发现性能优异的模型需大量内存与计算资源,超出标准部署基础设施支持能力。本研究扩展了表格数据分布外评估基准,为高风险领域中应用这些模型提供了依据。

原文摘要 · Abstract (English)

Tabular Foundation Models (TFMs) have emerged as novel approaches for tabular predictive tasks, demonstrating competitive predictive performance to ensemble tree-based models. Most TFMs are trained and evaluated on independent and identically distributed data, but this assumption changes in real-world scenarios due to distribution shifts, which compromise the robustness of models. Limited research has been conducted of TFMs under distribution shifts. We present an empirical evaluation of Out-Of-Distribution (OOD) performance of nine TFMs, spanning diverse pre-training strategies and architectures: TabPFNv2, TabPFNv2.5, TabPFNv2.6, TabPFNv3, TabICL, TabICLv2, Mitra, LimiX and TabFM. Three real-world datasets from the TableShift study were considered (HELOC, Voting, Childhood Lead), covering label, socioeconomic, and geographic shift types. Our results show that all evaluated TFMs degrade systematically under distribution shift regardless of pre-training strategy, with shift gaps ranging from 0.003 to 0.060 depending on shift type. The relationship between in-distribution and OOD predictive performance documented for classical tabular models extends into TFMs. We also identified a scalability gap, as high-performing models demand significant memory and computational resources beyond what standard deployment infrastructure can support. This study extends existing benchmarks for OOD in tabular data, providing evidence to support their adoption in high-stakes domains characterized by structural distribution shifts.

表格模型分布外鲁棒性评估

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。