arXiv:2607.11007cs.LG2026-07

TabPFN能精准校准多模态嵌入的置信度,且不损失分类准确率。

TabPFN beyond Tabular Data: Calibration and Accuracy on Multimodal Embeddings

论文配图:TabPFN beyond Tabular Data: Calibration and Accuracy on Multimodal Embeddings
图 1 · 摘自论文原文
  • 用TabPFN作零梯度分类头,直接适配冻结的多模态编码器。
  • 在22,820次测试中,NLL降低48%-62%,ECE提升2.1-5.3倍。
  • 适合中高样本量、低中维特征场景,尤其对置信度敏感任务有用。

少样本多模态分类通常在冻结的预训练编码器上附加轻量级分类头(如kNN、逻辑回归或线性SVM),虽计算高效,但置信度估计常欠校准。本文系统评估了将TabPFN作为冻结图像、文本和音频编码器的零梯度分类头,在14个数据集、11种编码器、三种模态下共22,820次评估中,其在负对数似然(NLL)和期望校准误差(ECE)上均取得最佳平均排名。在代表性设置下,相比八种基线平均值,其NLL下降48%-62%,ECE提升2.1-5.3倍,同时保持或超越基线准确率。校准优势具有广泛迁移性,而准确率优势仅在中高样本量(k≥50)和低中维特征(d≤32)时显著;当标签数据稀缺、特征维度高或基线逼近天花板时,优势减弱。在主干网络微调后,用TabPFN替换训练好的线性头,可提升校准性能并保持竞争力,表明表示适应与可靠头部选择是互补的。结果明确了TabPFN作为校准敏感型多模态分类的无训练头部的适用条件。代码与配置已公开于GitHub:https://github.com/Jingxiang-Zhang/tabpfn-multimodal-embeddings。

原文摘要 · Abstract (English)

Few-shot multimodal classification commonly attaches a lightweight head, such as $k$-nearest neighbors, logistic regression, or a linear SVM, to a frozen pretrained encoder. Although computationally efficient, these heads can produce poorly calibrated confidence scores. We ask whether TabPFN can provide reliable confidence estimates on multimodal embeddings without sacrificing predictive accuracy, and under what conditions. We systematically evaluate TabPFN as a zero-gradient head for frozen image, text, and audio encoders. Across 22{,}820 evaluation episodes spanning 14 datasets, 11 encoders, and three modalities, TabPFN achieves the best mean rank among nine classification heads on both negative log-likelihood (NLL) and expected calibration error (ECE). At a representative setting, it reduces NLL by 48--62\% and ECE by 2.1--5.3$\times$ relative to the average of eight baselines while matching or exceeding their average accuracy. This calibration benefit transfers broadly, whereas the accuracy advantage is conditional: it concentrates at moderate-to-high shot counts and low-to-moderate feature dimensions ($k \ge 50$, $d \le 32$), and diminishes when labeled data are scarce, feature dimensions are high, or competing methods approach ceiling accuracy. After backbone adaptation, replacing the trained linear head with TabPFN improves calibration while preserving competitive accuracy, showing that representation adaptation and reliable head choice are complementary. Together, these results identify when TabPFN can serve as a training-free head for calibration-sensitive multimodal classification. To support transparency and reproducibility, we publicly release the source code, experiment configurations, and evaluation scripts in our GitHub repository: https://github.com/Jingxiang-Zhang/tabpfn-multimodal-embeddings.

多模态置信度校准少样本学习TabPFN

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。