TabPack通过自动采样超参数，一键实现高效表格式深度学习集成模型。
TabPack: Efficient Hyperparameter Ensembles for Tabular Deep Learning
- 并行采样多种超参数配置的MLP，训练时动态筛选最优成员。
- 默认设置下性能媲美精调基线，节省大量调参时间和算力。
- 适合追求快速部署、不想繁琐调参的研究者与工程师。
在表格式数据的深度学习中，高效的多层感知机（MLP）集成近期成为有效且实用的架构。现有方法对所有底层MLP使用相同超参数，需进行超参数调优才能获得最佳性能。本文提出TabPack，一种具备强开箱即用性能的高效MLP集成方法，显著降低对传统调参的依赖。TabPack在单次运行中能高效并行采样并训练多个具有不同超参数的MLP，并在训练过程中动态选择集成成员。因此，仅需指定超参数采样范围，无需精确设定具体值，即可获得良好性能。在中大型公开数据集上的实验表明，使用默认配置的TabPack性能与经过广泛调优的先前方法相当，大幅减少了达到竞争性结果所需的努力和计算资源。值得注意的是，在现代MacBook上运行默认配置的TabPack所耗时间，甚至短于某些基线在工业级GPU上进行调参的时间。
原文摘要 · Abstract (English)
In deep learning for tabular data, efficient ensembles of multilayer perceptrons (MLPs) have recently emerged as effective and practical architectures. Existing methods of this kind use the same hyperparameters for all underlying MLPs, which requires hyperparameter tuning for achieving the best performance. In this work, we introduce TabPack, an efficient MLP ensemble with strong out-of-the-box performance and reduced reliance on traditional tuning. In a single run, TabPack samples and trains many MLPs with different hyperparameters efficiently in parallel and selects ensemble members on the fly during training. Thus, TabPack only requires specifying ranges from which to sample MLP hyperparameter rather than exact hyperparameter values, which naturally demands less precision for good performance. In experiments on medium-to-large public datasets, TabPack with default settings performs on par with extensively tuned prior methods, thus substantially reducing effort and compute resources needed to achieve competitive results on tabular tasks. Notably, running the default TabPack configuration on a modern MacBook took less time than tuning some baselines on an industry-grade GPU.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。