提出无监督表格式上下文选择方法,提升低标签场景下模型性能。
LUCoS: Latent Unsupervised Context Selection for Tabular Foundation Models
- 用无监督嵌入空间替代原始特征空间进行几何选择
- 在67个数据集上均优于随机选择,小预算下提升覆盖率,大预算下依赖表示空间质量
- 适合需要高效标注的低标签表格学习任务
在低标签表格学习中,选择哪些样本进行标注是关键挑战。对于近期的表格基础模型(如TabPFN),上下文选择直接决定预测性能。监督基准实验表明,在相同标注预算下,精心挑选的标注集可显著超越随机选择。然而,冷启动场景(即尚未有标签时就需选择样本)在表格基础模型研究中鲜受关注。该问题本质上是几何性质的:在视觉和语言领域,基础模型生成的嵌入空间支持有效的几何选择方法;而现有表格实例选择多在原始表格空间中进行,因特征类型异质、尺度混合及非线性交互,原始空间距离不可靠,导致原始空间选择在多数数据集上随预算增加反而劣于随机。本文提出LUCoS(Latent Unsupervised Context Selection),利用无监督先验拟合网络(PFN)生成的嵌入空间定义潜在几何,并选择代表性中位点作为上下文。在67个OpenML-CC18数据集上,六种低标签预算下,LUCoS在平均AUC、ACC和F1指标上均排名第一,结果在不同指标与数据集稳健性检验中保持一致。增益分解显示:小预算时主要优势来自覆盖度保障;预算增大后,决定性因素变为覆盖度测量所依赖的表示空间质量。LUCoS克服了原始特征空间选择的失败,表明可靠的无监督上下文选择不依赖选择器复杂度,而在于在有意义的表示几何中定义代表性。
原文摘要 · Abstract (English)
Selecting which instances to label is a key challenge in low-label tabular learning. For recent Tabular Foundation Models such as TabPFN, context selection directly determines predictive performance. Supervised oracle experiments show that carefully chosen labeled context sets can strongly outperform random selection under the same labeling budget. However, the cold-start setting, where instances must be selected before any labels are available, has received little attention in the TFM literature. This problem is fundamentally geometric. In vision and language, foundation models induce embedding spaces where simple geometric selection methods are effective. In contrast, tabular instance selection has so far been performed predominantly in the original tabular space, which lacks a natural metric; heterogeneous types, mixed scales, and nonlinear interactions make raw-space distances unreliable for context construction, and original-space selection falls below random on the majority of datasets as the budget grows. We propose LUCoS (Latent Unsupervised Context Selection), which replaces raw-feature geometry with the latent geometry induced by embeddings from an unsupervised Prior-Fitted Network (PFN) and selects representative medoids as context. Evaluated on 67 OpenML-CC18 datasets across six low-label budgets, LUCoS ranks first under mean AUC, ACC, and F1, with conclusions stable across metrics and dataset-level robustness checks. A gain decomposition reveals a simple mechanism: at the smallest budgets, the main benefit comes from enforcing coverage; as the budget increases, the decisive factor becomes the representation space in which coverage is measured. LUCoS mitigates failures of original feature space selection, showing that reliable unsupervised context selection depends less on selector sophistication than on defining representativeness in a meaningful representation geometry.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。