arXiv:2505.18125cs.LGcs.CL2025-05NeurIPS被引 15

让表格数据中的文本字段更智能,提升分类准确率。

TabSTAR: A Tabular Foundation Model for Tabular Data with Text Fields

  • 用目标感知的文本表示,动态生成任务相关嵌入
  • 在中大型数据集上超越现有方法,表现接近最优
  • 可跨数据集迁移,适合含文本的表格分类场景

尽管深度学习在多个领域取得显著进展,但在表格学习任务上表现仍逊于梯度提升决策树。近年来,具备真实世界知识并能跨多样化数据集泛化的表格基础模型正逐步兴起,尤其在包含自由文本的数据上优势明显。然而,现有方法多采用静态、与目标无关的文本表征,限制了效果。我们提出TabSTAR:一种具有语义目标感知表示的表格基础模型。该模型通过解冻预训练文本编码器,并输入目标标记以提供上下文,实现任务特定嵌入的学习,其架构不含数据集特异性参数。在含文本特征的分类任务基准测试中,TabSTAR在中等和大型数据集上均达到当前最优性能;其预训练阶段表现出数据集数量的缩放规律,为性能进一步提升提供路径。

原文摘要 · Abstract (English)

While deep learning has achieved remarkable success across many domains, it has historically underperformed on tabular learning tasks, which remain dominated by gradient boosting decision trees. However, recent advancements are paving the way for Tabular Foundation Models, which can leverage real-world knowledge and generalize across diverse datasets, particularly when the data contains free-text. Although incorporating language model capabilities into tabular tasks has been explored, most existing methods utilize static, target-agnostic textual representations, limiting their effectiveness. We introduce TabSTAR: a Tabular Foundation Model with Semantically Target-Aware Representations. TabSTAR is designed to enable transfer learning on tabular data with textual features, with an architecture free of dataset-specific parameters. It unfreezes a pretrained text encoder and takes as input target tokens, which provide the model with the context needed to learn task-specific embeddings. TabSTAR achieves state-of-the-art performance for both medium- and large-sized datasets across known benchmarks of classification tasks with text features, and its pretraining phase exhibits scaling laws in the number of datasets, offering a pathway for further performance improvements.

表格数据文本融合基础模型迁移学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。