系统梳理表格数据表示学习方法,分类解析模型设计与应用场景。
Representation Learning for Tabular Data: A Comprehensive Survey
- 按泛化能力分为专用、可迁移和通用三类模型
- 提出特征与样本级表示学习的层级分类体系
- 适合从事表格数据建模与跨领域迁移研究的读者
表格数据以行列表形式广泛存在于机器学习分类与回归任务中。近年来,深度神经网络(DNN)凭借其表示学习能力展现出良好效果。本文系统综述表格表示学习领域,涵盖背景、挑战与基准,分析DNN在该领域的优劣。将现有方法按泛化能力分为三类:专用模型聚焦于同一数据分布内的训练与评估;可迁移模型在单个或多个数据集上预训练后微调至下游任务,利用同源或异源知识,甚至跨模态(如视觉与语言)信息;通用模型(又称表格基础模型)进一步扩展,无需微调即可直接应用于下游任务。我们根据跨异构数据集的适应策略对通用模型进行分组。此外,探讨集成方法融合多模型优势。最后讨论代表性延伸方向:开放环境表格机器学习、表格与多模态融合、表格理解等。更多信息见:https://github.com/LAMDA-Tabular/Tabular-Survey。
原文摘要 · Abstract (English)
Tabular data, structured as rows and columns, is among the most prevalent data types in machine learning classification and regression applications. Models for learning from tabular data have continuously evolved, with Deep Neural Networks (DNNs) recently demonstrating promising results through their capability of representation learning. In this survey, we systematically introduce the field of tabular representation learning, covering the background, challenges, and benchmarks, along with the pros and cons of using DNNs. We organize existing methods into three main categories according to their generalization capabilities: specialized, transferable, and general models. Specialized models focus on tasks where training and evaluation occur within the same data distribution. We introduce a hierarchical taxonomy for specialized models based on the key aspects of tabular data -- features, samples, and objectives -- and delve into detailed strategies for obtaining high-quality feature- and sample-level representations. Transferable models are pre-trained on one or more datasets and subsequently fine-tuned on downstream tasks, leveraging knowledge acquired from homogeneous or heterogeneous sources, or even cross-modalities such as vision and language. General models, also known as tabular foundation models, extend this concept further, allowing direct application to downstream tasks without fine-tuning. We group these general models based on the strategies used to adapt across heterogeneous datasets. Additionally, we explore ensemble methods, which integrate the strengths of multiple tabular models. Finally, we discuss representative extensions of tabular learning, including open-environment tabular machine learning, multimodal learning with tabular data, and tabular understanding. More information can be found in the following repository: https://github.com/LAMDA-Tabular/Tabular-Survey.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。