集成表格式基础模型效果有限，因模型间冗余度高，且易破坏预测校准性。
Ensembling Tabular Foundation Models - A Diversity Ceiling And A Calibration Trap
- 六种表格式模型高度相似，平均成对相关性达0.961，集成空间受限。
- 最优集成方法仅提升0.18%准确率，但需253倍计算成本，性价比低。
- 逻辑回归元学习器虽提升准确率，却严重破坏校准性，适合追求精度者。
表格式基础模型（TFMs）在越来越多的表格任务上达到或超越调优后的梯度提升树表现，但无单一模型在所有数据集上胜出。集成是常用解决方案，但效果远低于预期。六种现代TFMs构成几乎冗余的集合：其平均成对Q统计量为0.961，接近1，导致任意凸组合均有上限。我们在153个OpenML分类任务上评估六种集成策略与六种TFMs。最佳方法为两层级堆叠集成，相比最强单模型仅提升0.18%准确率，但计算开销增加253倍。Friedman与Nemenyi分析显示，三种集成与最优基线模型处于同一等效组；其余三种集成显著劣于最优基线。以逻辑回归为元学习器的堆叠方法表现突出：准确率与ROC-AUC具竞争力，但对数损失排名最差。该方法通过锐化类别边界提升准确率，却损害了预测校准性。建议采用贪婪选择作为实际默认方案。
原文摘要 · Abstract (English)
Tabular foundation models (TFMs) now match or beat tuned gradient-boosted trees on a growing fraction of tabular tasks, but no single TFM wins on every dataset. Ensembling is the go to fix here, and it works less well than expected. Six modern TFMs form a near-redundant pool: their mean pairwise Q-statistic is $0.961$, close enough to $1$ that any convex combination is bounded above. We benchmark six ensemble strategies over six TFMs on 153 OpenML classification tasks. The best ensemble, two-level cascade stacking, buys $+0.18\%$ accuracy over the strongest single TFM at $253\times$ the compute. A Friedman and Nemenyi analysis places three ensembles and the best base TFM in a single equivalence group; three other ensembles are significantly \emph{worse} than the best base. Stacking with a logistic-regression meta-learner is the most striking case: competitive accuracy and ROC-AUC, the worst log-loss rank among the ensembles. The meta-learner improves accuracy by sharpening class boundaries, which destroys calibration. We recommend greedy selection as the practical default.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。