比较表格模型的数据先验,发现不同先验影响下游表现和稳定性。
Towards Evaluating Data Priors for Tabular Foundation Models

- 统一接口生成任务,用相同模型和训练流程测试不同先验
- 不同先验导致性能差异,有的绝对表现强,有的排序更稳定
- 数据相似性只能部分解释下游行为,先验作用不可忽视
数据生成先验是表格基础模型的核心组件,决定了预训练时的任务分布。然而,先验很少被独立评估,难以理解其对下游模型行为的影响。本文提出一种统一接口,集成近期表格基础模型的公开先验及真实数据构建的先验。从每个先验生成训练任务,在固定架构与训练协议下训练同一模型,并在共享下游分类任务上评估性能。通过生成任务统计与下游预测表现对比先验。结果表明,不同先验偏好不同下游行为:部分先验实现更强绝对性能,另一些则在多个数据集上保持更一致的相对排名。进一步发现,数据层面相似性仅能部分解释下游表现差异。代码已开源:https://github.com/automl/TFM-Playground/tree/prior-dev。
原文摘要 · Abstract (English)
Data-generating priors are a central component of tabular foundation models because they define the task distribution used during pretraining. However, priors are rarely evaluated as independent components, making it difficult to understand how much they affect downstream model behavior. This raises a methodological question: how can priors from different tabular foundation models be compared independently of the architectures and training protocols they were introduced with? To study this question, we implement a unified interface for publicly available priors from recent tabular foundation models and priors constructed from real datasets. We generate training tasks from each prior, train the same model architecture under a fixed training protocol, and evaluate the resulting models on shared downstream classification tasks. We compare priors through both generated-task statistics and downstream predictive performance. Our results show that different priors favor different downstream behaviors, with some achieving stronger absolute performance and others exhibiting more consistent relative rankings across datasets. We further find that data-level similarity only partially explains downstream behavior. Our code is available at https://github.com/automl/TFM-Playground/tree/prior-dev.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。