arXiv:2512.12405cs.LG2025-12

给表格模型加个简单图结构,就能显著提升预测效果。

Can Graphs Improve Tabular Foundation Models?

  • 用二部图连接样本与特征值锚点,轻量GNN增强行表示。
  • 在80个分类和64个回归任务中胜过多个强基线模型。
  • 适合想提升表格预训练模型性能的研究者或工程师。

表格数据在现实系统中至关重要。尽管近期的表格变压器(如SAINT、TP-BERTa、TabPFN、TabICL、MITRA)已引入有限的行间推理能力,但大多数方法仍缺乏显式建模实例间关系的机制,而相似样本常有相关结果。本文研究引入简单图先验是否能增强预训练表格变压器。提出BOLERO,一种轻量级静态二部图头,用于增强基于掩码标记预测预训练的RoBERTa-Tab(一种罗伯特风格的表格主干网络)。每个实例连接至特征/值锚点;小型图神经网络(GNN)优化行表示,而主干网络保持冻结。在来自TP-BERTa基准套件的80个分类和64个回归数据集上评估,对比包括XGBoost、CatBoost、TabPFN-v2、MITRA、TabICL、TP-BERTa和RoBERTa-Tab在内的强基线。采用配对威尔科克森符号秩检验及效应量(中位数改进与置信区间),确保结论统计可靠。BOLERO在分类与回归任务中均取得最多的统计显著胜出次数,证明轻量图先验可有效提升预训练表格变压器性能。

原文摘要 · Abstract (English)

Tabular data are central to many real-world systems. While recent tabular transformers and in-context learners such as SAINT, TP-BERTa, TabPFN, TabICL, and MITRA incorporate limited inter-row reasoning, most approaches still lack an explicit mechanism to model relationships among instances, even though similar samples often share related outcomes. We investigate whether introducing \emph{simple graph priors} can enhance \emph{pretrained tabular transformers}. Concretely, we introduce {BOLERO}, a lightweight, static bipartite graph head that augments {RoBERTa-Tab} (a RoBERTa-style tabular backbone pretrained with masked-token prediction.) Each instance connects to feature/value anchors; a small GNN refines row representations, while the backbone remains frozen. We evaluate on 80 classification and 64 regression datasets from the TP-BERTa benchmark suites, comparing against strong baselines including XGBoost, CatBoost, TabPFN-v2, MITRA, TabICL, TP-BERTa, and RoBERTa-Tab. To ensure statistically sound conclusions, we follow best practices for multi-dataset evaluation: pairwise Wilcoxon signed-rank tests on per-dataset score differences and effect sizes (median improvement with confidence intervals), rather than mean-rank post-hoc tests that depend on the competitor pool. BOLERO achieves the highest number of statistically significant wins across both classification and regression, demonstrating that lightweight graph priors meaningfully improve pretrained tabular transformers.

表格模型图神经网络预训练机器学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。