提出三种平滑基模型,挑战树模型在表格回归中的主导地位。
Revisiting Chebyshev Polynomial and Anisotropic RBF Models for Tabular Regression
- 设计自适应中心与梯度优化宽度的各向异性RBF网络
- 在55个数据集上平滑模型与树模型精度相当但泛化差距更小
- 适合需要稳定预测和良好泛化能力的科研与工业场景
光滑基模型如切比雪夫多项式回归器和径向基函数(RBF)网络在数值分析中已有广泛应用,其连续可微的预测表面适用于代理优化、敏感性分析等输入变化缓慢的场景。尽管如此,这类模型在表格回归中极少被采用,主流仍是树集成方法。本文在55个按应用领域组织的回归数据集上对比多种模型,提出一种数据驱动中心放置的各向异性RBF网络、带岭正则化的切比雪夫多项式回归器,以及一个切比雪夫模型树混合结构,三者均以scikit-learn兼容形式开源。实验表明,预训练Transformer在多数数据集上精度最高,但受制于GPU依赖、推理延迟和数据量限制,难以在基于CPU的科研与工业场景部署。在仅需CPU的模型中,平滑模型与树模型精度统计上无显著差异,但前者通常表现出更小的泛化差距。建议在实际任务中将平滑基模型纳入候选池,尤其当下游任务需要紧密泛化和渐变预测时。
原文摘要 · Abstract (English)
Smooth-basis models such as Chebyshev polynomial regressors and radial basis function (RBF) networks are well established in numerical analysis. Their continuously differentiable prediction surfaces suit surrogate optimisation, sensitivity analysis, and other settings where the response varies gradually with inputs. Despite these properties, smooth models seldom appear in tabular regression, where tree ensembles dominate. We ask whether they can compete, benchmarking models across 55 regression datasets organised by application domain. We develop an anisotropic RBF network with data-driven centre placement and gradient-based width optimisation, a ridge-regularised Chebyshev polynomial regressor, and a smooth-tree hybrid (Chebyshev model tree); all three are released as scikit-learn-compatible packages. We benchmark these against tree ensembles, a pre-trained transformer, and standard baselines, evaluating accuracy alongside generalisation behaviour. The transformer ranks first on accuracy across a majority of datasets, but its GPU dependence, inference latency, and dataset-size limits constrain deployment in the CPU-based settings common across applied science and industry. Among CPU-viable models, smooth models and tree ensembles are statistically tied on accuracy, but the former tend to exhibit tighter generalisation gaps. We recommend routinely including smooth-basis models in the candidate pool, particularly when downstream use benefits from tighter generalisation and gradually varying predictions.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。