研究不同样条编码对表格数据深度学习的性能影响,发现可学习节点能提升效果但增加训练成本。
From Uniform to Learned Knots: A Study of Spline-Based Numerical Encodings for Tabular Deep Learning

- 采用可微分节点参数化,实现端到端优化样条节点位置。
- 分类任务中分段线性编码最稳健,回归任务则依赖具体编码与模型结构。
- 适合关注表格数据特征工程与计算效率权衡的研究者使用。
数值预处理仍是表格深度学习的关键环节,连续特征的表示方式显著影响下游性能。本文系统研究了基于样条的数值编码方法,包括B样条、M样条和积分样条(I样条),在均匀、分位数、目标感知及可学习节点布局下的表现。针对可学习变体,采用可微分节点参数化,实现节点位置与主干网络的稳定联合优化。在多种公开回归与分类数据集上,使用MLP、ResNet和FT-Transformer主干进行评估,并与常见预处理基线对比。结果表明,编码有效性高度依赖任务类型、编码大小与主干网络。分类任务中分段线性编码(PLE)总体最稳健,样条编码仍具竞争力;回归任务无单一主导编码,性能取决于样条族与节点布局策略,且大模型(如MLP、ResNet)收益更明显,而FT-Transformer提升较小。可学习节点编码可稳定优化,但显著增加训练开销。因此,数值编码应综合考虑预测性能与计算成本。代码已开源:https://github.com/mkumar73/tdl-numerical-encodings/。
原文摘要 · Abstract (English)
Numerical preprocessing remains a critical component of tabular deep learning, as the representation of continuous features can strongly affect downstream performance. We systematically study spline-based numerical encodings, including B-splines, M-splines, and integrated splines (I-splines), under uniform, quantile-based, target-aware, and learnable-knot placement. For the learnable variants, we adopt a differentiable knot parameterization that enables stable end-to-end optimization of knot locations jointly with the backbone. We evaluate these encodings on a diverse collection of public regression and classification datasets using MLP, ResNet, and FT-Transformer backbones, and compare them against common numerical preprocessing baselines. Our results show that the effectiveness of numerical encoding depends strongly on the task, encoding size, and backbone. For classification, piecewise-linear encoding (PLE) is the most robust choice overall, while spline-based encodings remain competitive. For regression, no single encoding dominates, with performance depending on the spline family and knot-placement strategy, and larger gains generally observed for MLP and ResNet than for FT-Transformer. Learnable-knot variants can be optimized stably but may substantially increase training cost. Overall, numerical encodings should therefore be assessed jointly in terms of predictive performance and computational overhead. The implementation is publicly available at https://github.com/mkumar73/tdl-numerical-encodings/.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。