用大模型将表格数据转为文本，实现高效特征编码。
LLM Embeddings for Deep Learning on Tabular Data
- 将表格数据转为文本，用大模型生成嵌入向量
- 在7个分类数据集上超越MLP、ResNet等基线模型
- 适合希望复用预训练知识的表格深度学习研究者
表格深度学习方法需将数值和类别特征嵌入高维空间。现有方法采用针对不同类型的独立编码方式，限制了跨表迁移能力和预训练知识利用。本文提出新方法：先将表格数据转化为文本，再利用大语言模型的预训练表示进行编码，形成即插即用的表格深度学习解决方案。在七个分类数据集上的实验表明，该方法在准确率上优于MLP、ResNet和FT-Transformer等竞争模型。
原文摘要 · Abstract (English)
Tabular deep-learning methods require embedding numerical and categorical input features into high-dimensional spaces before processing them. Existing methods deal with this heterogeneous nature of tabular data by employing separate type-specific encoding approaches. This limits the cross-table transfer potential and the exploitation of pre-trained knowledge. We propose a novel approach that first transforms tabular data into text, and then leverages pre-trained representations from LLMs to encode this data, resulting in a plug-and-play solution to improv ing deep-learning tabular methods. We demonstrate that our approach improves accuracy over competitive models, such as MLP, ResNet and FT-Transformer, by validating on seven classification datasets.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。