提出首个原生理解表格数据的通用模型,无需预处理即可直接分析原始数据。
Data Language Models: A New Foundation Model Class for Tabular Data
- 构建直接解析原始单元格值的表格语言模型,摆脱传统预处理流程。
- 在行级预测和缺失值重建任务中超越梯度提升与主流基础模型表现。
- 可仅凭原始数据识别未知数据集所属行业,适用于各类垂直领域应用。
每种主要数据模态如今都有其原生理解的基础模型:文本有语言模型,图像有视觉模型,音频有音频模型。然而,许多重要现实世界AI决策所依赖的表格数据却缺乏此类基础模型。当前所有表格AI方法,从梯度提升树到最新表格基础模型,均需经过预处理管道才能使用数据,且均不真正理解表格作为模态的本质。本文提出数据语言模型(DLM),填补了表格数据领域的基础模型空白。DLM以语言模型理解句子的方式原生理解表格:无需序列化或预处理,直接从原始单元格值输入。它是构建AI模型、智能体及垂直应用的表格数据底层层,消除了现有系统中阻碍原始数据与AI之间的预处理管道。我们提出首个DLM——Schema-1,一个拥有1.4亿参数、在超过230万合成与真实世界表格数据集上训练的模型。Schema-1在多个标准行级预测基准上优于梯度提升集成、AutoML堆栈及评估中的其他表格基础模型。在缺失值重建任务中,其平均性能低于所有经典统计方法,并优于前沿大语言模型,表明对数据自身分布几何结构的理解比语言模型中的世界知识更有利于插补。此外,它能仅凭原始数据可靠识别任意未见数据集的行业类别,这是以往任何表格模型无法实现的任务。Schema-1是人工智能体系中缺失已久的原生表格理解层。
原文摘要 · Abstract (English)
Every major data modality now has a foundation model that understands it natively: text has language models, images have vision models, audio has audio models. Tabular data, the modality on which many consequential real-world AI decisions are made, does not. Every approach to tabular AI today, from gradient-boosted trees to the latest tabular foundation models, requires a preprocessing pipeline before any model can consume the data. None of them understand tabular data as a modality. We introduce the Data Language Model (DLM), the missing foundation model for tabular data. A DLM understands tables the way a language model understands sentences: natively, without serialization or preprocessing, directly from raw cell values. It is the tabular data layer on which AI models, agents, and vertical AI applications can be built, eliminating the preprocessing pipelines that currently stand between raw data and every AI system that consumes it. We present Schema-1, the first DLM: a 140M parameter model trained on more than 2.3M synthetic and real-world tabular datasets. Schema-1 outperforms gradient-boosted ensembles, AutoML stacks, and the tabular foundation models we evaluate on established row-level prediction benchmarks. On missing value reconstruction it achieves lower reconstruction error than all classical statistical methods and frontier large language models on mean performance across conditions, establishing that structural understanding of a dataset's own distributional geometry is more useful for imputation than world knowledge encoded in language. It identifies the industry sector of any unseen dataset from raw cell values alone, reliably across any domain, a task no prior tabular model can perform. It is the native tabular understanding layer that has been missing from the AI stack.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。