评测发现表格基础模型可直接用于回归中的分布估计,效果优于传统方法。
Benchmarking Tabular Foundation Models for Conditional Density Estimation in Regression
- 用39个真实数据集测试三种表格基础模型在条件密度估计中的表现。
- 在多数数据集上,基础模型在密度损失、对数似然和CRPS上领先。
- 适合需要快速部署分布预测的科研与工业场景。
条件密度估计(CDE)——从表格协变量中恢复响应变量的完整条件分布——在异方差性、多模态或非对称不确定性场景中至关重要。近年来的表格基础模型(如TabPFN和TabICL)能自然生成预测分布,但其作为通用CDE方法的有效性尚未系统评估,与点预测性能相比研究不足。我们在39个真实世界数据集上,对三种表格基础模型变体与多种参数化、树基和神经网络基线进行对比,训练样本量从50到20,000不等,使用六项指标涵盖密度精度、校准性和计算时间。在所有样本量下,基础模型在绝大多数数据集上取得最佳的CDE损失、对数似然和CRPS。校准性在小样本时表现良好,但在大样本时部分指标落后于专用神经基线,提示后处理校准可能有益。在使用SDSS DR18的光度红移案例研究中,仅用50,000个训练星系的TabPFN,表现超过在全部500,000个星系上训练的所有基线。综合结果表明,表格基础模型是强大的开箱即用型条件密度估计器。
原文摘要 · Abstract (English)
Conditional density estimation (CDE) - recovering the full conditional distribution of a response given tabular covariates - is essential in settings with heteroscedasticity, multimodality, or asymmetric uncertainty. Recent tabular foundation models, such as TabPFN and TabICL, naturally produce predictive distributions, but their effectiveness as general-purpose CDE methods has not been systematically evaluated, unlike their performance for point prediction, which is well studied. We benchmark three tabular foundation model variants against a diverse set of parametric, tree-based, and neural CDE baselines on 39 real-world datasets, across training sizes from 50 to 20,000, using six metrics covering density accuracy, calibration, and computation time. Across all sample sizes, foundation models achieve the best CDE loss, log-likelihood, and CRPS on the large majority of datasets tested. Calibration is competitive at small sample sizes but, for some metrics and datasets, lags behind task-specific neural baselines at larger sample sizes, suggesting that post-hoc recalibration may be a valuable complement. In a photometric redshift case study using SDSS DR18, TabPFN exposed to 50,000 training galaxies outperforms all baselines trained on the full 500,000-galaxy dataset. Taken together, these results establish tabular foundation models as strong off-the-shelf conditional density estimators.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。