用轻量预处理提升表格数据模型表现,填补了基准测试中的特征工程空白。
TabPrep: Closing the Feature Engineering Gap in Tabular Benchmarks

- 设计针对三种数据结构模式的特征生成器,实现系统化预处理。
- 在TabArena上显著提升树、神经网络等各类模型性能,超越纯模型优化。
- 开源工具支持大规模基准测试,适合关注真实场景建模的研究者。
表格机器学习的进步主要集中在更复杂的模型架构上,而特征工程作为实际建模流程中的关键环节,在现代基准测试中却完全缺失,导致评估结果存在未量化差距。本文提出TabPrep,一个轻量级预处理管道,包含针对三种特定数据结构模式精心设计的特征生成器。我们发现,许多广泛使用的模型类别对这些模式存在可预测的盲区,而系统性特征工程本身即可达到新性能峰值。在TabArena基准上,将TabPrep融入模型训练与调优,持续提升树模型、神经网络、线性模型及基础模型的表现,常优于仅靠模型改进带来的增益。相比以往自动化特征工程方法,TabPrep在性能、效率和跨数据集适用性上均更优,支持大规模基准集成。通过开源(见https://github.com/atschalz/tabprep),我们使研究者能将特征工程纳入基准设置,填补表格评估领域长期存在的空白。
原文摘要 · Abstract (English)
Progress in tabular machine learning has largely focused on increasingly sophisticated model architectures. At the same time, feature engineering remains a critical yet underexplored component of real-world modeling pipelines that is entirely absent from modern benchmarks, which creates an unquantified evaluation gap. In this work, we introduce TabPrep, a lightweight preprocessing pipeline composed of feature generators that are carefully designed to target three specific structural data patterns. We show that many widely used model classes exhibit predictable blind spots to these patterns and that systematic feature engineering alone can establish new peak performance. Across the TabArena benchmark, integrating TabPrep into model training and tuning consistently improves performance for tree-based, neural, linear, and foundation models, often surpassing gains achieved by model-centric innovations alone. TabPrep outperforms previous automated feature engineering approaches in performance, efficiency, and applicability across datasets, enabling integration into large-scale benchmarks. By releasing TabPrep (see https://github.com/atschalz/tabprep), we enable researchers to integrate feature engineering into their benchmarking setup, filling a longstanding gap in tabular evaluations.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。