用合理评分规则评估表格大模型的预测分布，避免只看平均值。
Distributional Regression with Tabular Foundation Models: Evaluating Probabilistic Predictions via Proper Scoring Rules
- 用严格合理的评分规则替代传统点估计指标
- 不同评分规则在有限数据下导致模型表现差异显著
- 可按任务需求调整模型训练目标，适合精准预测场景
现代表格基础模型（如 TabPFN、TabICL）天然生成完整的预测分布，但现有评估基准（如 TabArena、TALENT）仍主要依赖点估计指标（如 RMSE、$R^2$）。这种不匹配隐含奖励仅优化条件均值而忽视预测分布质量的模型。本文主张在训练、微调和基准评测中使用合理评分规则。尽管所有严格合理的评分规则在总体上等价，但在有限样本下可能引发不同的归纳偏置，导致性能差异。我们通过在多个数据集上对 TabPFN 与 TabICL 进行微调实验，验证了评分规则与评价指标之间的非平凡交互作用。结果表明，从业者可依据特定评分目标适配模型，评分规则的选择实际影响模型行为。
原文摘要 · Abstract (English)
Modern tabular foundation models such as TabPFN and TabICL naturally produce full predictive distributions, while the benchmarks used to evaluate them (TabArena, TALENT, and others) still rely almost exclusively on point-estimate metrics (RMSE, $R^2$). This mismatch implicitly rewards machine learning models or pipelines that elicit a good conditional mean while ignoring the quality of the predictive distribution. We make the case for using proper scoring rules for training, fine-tuning, and benchmarking (ranking) of tabular foundation models. Although all strictly proper scoring rules are theoretically equivalent at the population level, they may differ on finite data: We demonstrate analytically and empirically that different scoring rules can induce different inductive biases during finite-sample optimization, leading to different model performance. We validate this finding by running fine-tuning experiments with TabPFN and TabICL using different scoring rules for various data sets, revealing non-trivial interactions between training objectives and evaluation metrics. Our results show that practitioners can adapt tabular foundation models to task-specific scoring objectives, and that the choice of scoring rule can influence model behavior in practice.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。