用提升法构造随机特征网络，小中规模数据表现远超传统方法。
Random Feature Representation Boosting
- 每层用随机特征拟合梯度，逐层提升模型表达能力。
- 小中规模数据上显著优于随机特征网络和端到端残差网络。
- 计算高效且有理论保证，适合资源受限场景使用。
我们提出随机特征表示提升（RFRBoost），一种基于提升理论构建深度残差随机特征神经网络（RFNN）的新方法。RFRBoost 在每一层使用随机特征学习网络表示的函数梯度，在保持RFNN凸优化优势的同时提升性能。在均方误差损失下，可获得逐层贪婪提升的闭式解；对于一般损失函数，拟合随机特征残差块等价于求解一个二次约束最小二乘问题。在表格数据回归与分类任务的大量实验表明，RFRBoost在小至中等规模数据上显著优于标准RFNN和端到端训练的MLP ResNet。此外，该方法具备显著计算优势，并拥有来自提升理论的理论保障。
原文摘要 · Abstract (English)
We introduce Random Feature Representation Boosting (RFRBoost), a novel method for constructing deep residual random feature neural networks (RFNNs) using boosting theory. RFRBoost uses random features at each layer to learn the functional gradient of the network representation, enhancing performance while preserving the convex optimization benefits of RFNNs. In the case of MSE loss, we obtain closed-form solutions to greedy layer-wise boosting with random features. For general loss functions, we show that fitting random feature residual blocks reduces to solving a quadratically constrained least squares problem. Through extensive numerical experiments on tabular datasets for both regression and classification, we show that RFRBoost significantly outperforms RFNNs and end-to-end trained MLP ResNets in the small- to medium-scale regime where RFNNs are typically applied. Moreover, RFRBoost offers substantial computational benefits, and theoretical guarantees stemming from boosting theory.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。