解决表格模型在标签偏移下的过拟合问题，提升分类准确性。
Mitigating Label Shift in Tabular In-Context Learning via Test-Time Posterior Adjustment
- 测试时调整后验概率，降低训练先验影响
- 在250+数据集上显著提升标签偏移场景表现
- 无需修改模型结构或额外训练，适合实际部署
TabPFN作为表格数据的基座模型，通过合成数据的上下文学习取得了优异性能。然而我们发现其在标签偏移下容易过拟合多数类。为此提出DistPFN，首个针对表格基座模型的测试时后验调整方法：通过削弱训练先验（即上下文中的类别分布）影响、强化模型预测后验贡献，实现不修改架构且无需额外训练的调整。进一步提出DistPFN-T，引入温度缩放，根据先验与后验差异自适应调节调整强度。在超过250个OpenML数据集上评估，各类基于TabPFN的模型在标签偏移场景中均获得显著提升，同时在标准设置下保持原有强性能。代码已开源。
原文摘要 · Abstract (English)
TabPFN has recently gained attention as a foundation model for tabular datasets, achieving strong performance by leveraging in-context learning on synthetic data. However, we find that TabPFN is vulnerable to label shift, often overfitting to the majority class in the training dataset. To address this limitation, we propose DistPFN, the first test-time posterior adjustment method designed for tabular foundation models. DistPFN rescales predicted class probabilities by downweighting the influence of the training prior (i.e., the class distribution of the context) and emphasizing the contribution of the model's predicted posterior, without architectural modification or additional training. We further introduce DistPFN-T, which incorporates temperature scaling to adaptively control the adjustment strength based on the discrepancy between prior and posterior. We evaluate our methods on over 250 OpenML datasets, demonstrating substantial improvements for various TabPFN-based models in classification tasks under label shift, while maintaining strong performance in standard settings without label shift. Code is available at this repository: https://github.com/seunghan96/DistPFN.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。