构建首个含字符串的表格学习基准,揭示编码方式对模型性能的关键影响。
STRABLE: Benchmarking Tabular Machine Learning with Strings
- 构建108个真实世界表格数据集,覆盖多领域字符串与数值混合场景。
- 多数表格以分类变量为主,简单编码+传统模型即可高效预测。
- 自由文本主导的表格需大模型编码,且后处理策略影响显著。
表格学习基准研究已证明专用架构的优势,但现实表格常包含字符串而非仅数值,这类场景因缺乏可靠基准而被忽视。本文提出STRABLE,一个包含108个真实世界学习任务的基准数据集,涵盖多种应用领域的字符串与数值混合表格。我们首次开展大规模实证研究,评估445种学习管道,包括端到端架构与模块化流程(先编码字符串,再后处理,最后输入表格学习器)。结果表明:由于野外多数表格为分类主导,结合简单字符串嵌入的先进表格学习器可在低计算成本下实现良好预测;在自由文本主导的表格上,大型语言模型编码器表现更具竞争力,且其性能受后处理影响显著,不同语言模型家族间存在差异。此外,STRABLE能生成接近理想排名的可泛化管道排序,确立其作为字符串表格学习研究基础的地位。
原文摘要 · Abstract (English)
Benchmarking tabular learning has revealed the benefit of dedicated architectures, pushing the state of the art. But real-world tables often contain string entries, beyond numbers, and these settings have been understudied due to a lack of a solid benchmarking suite. They lead to new research questions: Are dedicated learners needed, with end-to-end modeling of strings and numbers? Or does it suffice to encode strings as numbers, as with a categorical encoding? And if so, do the resulting tables resemble numerical tabular data, calling for the same learners? To enable these studies, we contribute STRABLE, a benchmarking corpus of 108 tables, all real-world learning problems with strings and numbers across diverse application fields. We run the first large-scale empirical study of tabular learning with strings, evaluating 445 pipelines. These pipelines span end-to-end architectures and modular pipelines, where strings are first encoded, then post-processed, and finally passed to a tabular learner. We find that, because most tables in the wild are categorical-dominant, advanced tabular learners paired with simple string embeddings achieve good predictions at low computational cost. On free-text-dominant tables, large LLM encoders become competitive. Their performance also appears sensitive to post-processing, with differences across LLM families. Finally, we show that STRABLE is a good set of tables to study "string tabular" learning as it leads to generalizable pipeline rankings that are close to the oracle rankings. We thus establish STRABLE as a foundation for research on tabular learning with strings, an important yet understudied area.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。