将表格转为图与文本双模态,通过一致性学习提升特征表示能力。
TabGLM: Tabular Graph Language Model for Learning Transferable Representations Through Multi-Modal Consistency Minimization
- 将每行表数据转为图和文本,用GNN与文本编码器分别处理。
- 在25个基准数据集上平均AUC-ROC提升5.56%,超越现有最佳方法。
- 适合处理异构特征的表格数据,参数量少且可迁移性强。
表格数据中的异构特征给深度学习模型带来挑战。尽管注意力机制和自监督学习取得进展,但在表格数据上的表现仍不及线性与树模型。现有将表格转为图像、语言或图等单模态的方法,在特征异构情况下性能受限。为此,我们提出TabGLM(表格图语言模型),一种新型多模态架构,能同时建模表格的结构与语义信息。该模型将每行数据转换为全连接图与序列化文本,分别由图神经网络(GNN)和文本编码器处理,并通过联合多模态自监督目标对齐两种表示,从而融合互补信息以增强特征学习。其灵活的图-文本流水线在异构数据上高效运行,参数量显著低于现有深度学习方法。在25个基准数据集上的评估显示,TabGLM平均AUC-ROC提升达5.56%,超越当前最先进(SoTA)的表格学习方法。
原文摘要 · Abstract (English)
Handling heterogeneous data in tabular datasets poses a significant challenge for deep learning models. While attention-based architectures and self-supervised learning have achieved notable success, their application to tabular data remains less effective over linear and tree based models. Although several breakthroughs have been achieved by models which transform tables into uni-modal transformations like image, language and graph, these models often underperform in the presence of feature heterogeneity. To address this gap, we introduce TabGLM (Tabular Graph Language Model), a novel multi-modal architecture designed to model both structural and semantic information from a table. TabGLM transforms each row of a table into a fully connected graph and serialized text, which are then encoded using a graph neural network (GNN) and a text encoder, respectively. By aligning these representations through a joint, multi-modal, self-supervised learning objective, TabGLM leverages complementary information from both modalities, thereby enhancing feature learning. TabGLM's flexible graph-text pipeline efficiently processes heterogeneous datasets with significantly fewer parameters over existing Deep Learning approaches. Evaluations across 25 benchmark datasets demonstrate substantial performance gains, with TabGLM achieving an average AUC-ROC improvement of up to 5.56% over State-of-the-Art (SoTA) tabular learning methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。