用残差下降准则逐步构建隐藏层,提升随机神经网络性能
Residual-Guided Randomized Neural Networks

- 通过残差减少量筛选随机生成的隐藏单元,逐轮优化
- 在71个数据集上准确率、稳定性均优于基线模型
- 方法通用性强,适合各类神经网络架构快速训练
随机神经网络通过固定输入到隐藏层的参数并以闭式解学习输出权重,实现快速且可解析的训练;然而其性能严重依赖一次无指导的隐藏单元采样,常导致冗余表示和模型容量利用不足。为解决此问题,本文提出一种简单且普适的残差引导方法:贪婪地基于闭式残差下降准则构建隐藏层。每一步包括:(i) 生成一组随机候选单元,(ii) 按岭正则化目标函数的精确下降量评分,(iii) 选取前k个最优单元,(iv) 使用标准设计(含直接输入连接)闭式重拟合读出层。该过程保证训练目标单调递减。方法模型无关:仅候选生成与架构相关,评分-选择-重拟合循环通用。在来自UCI仓库的71个基准数据集(涵盖二分类与多分类任务)上的大量实验表明,所提残差引导模型在准确率、稳定性和整体排名表现上持续优于基线。
原文摘要 · Abstract (English)
Randomized neural networks enable fast and analytically tractable training by fixing the input to hidden layer parameters at random and learning the output weights in closed form; however, their performance critically depends on a single uninformed draw of hidden units. This one shot and task uninformed feature construction often leads to redundant representations and suboptimal utilization of model capacity. To address this limitation, we propose a simple and broadly applicable residual guided procedure that greedily constructs the hidden layer using a closed form residual decrease criterion. At each stage, we (i) generate a pool of random candidate units, (ii) score each candidate by the exact reduction it induces in the ridge regularized objective, (iii) select the top k units, and (iv) refit the readout in closed form using the standard design with direct input links. This procedure yields a progressive training process with a guaranteed monotonic decrease of the training objective. The method is model agnostic: only the candidate generation is architecture specific, while the scoring selection refitting loop is shared across models. Extensive experiments on 71 benchmark datasets from the UCI repository, covering both binary and multiclass classification tasks, demonstrate that the proposed residual-guided models consistently outperform their baseline counterparts in terms of accuracy, stability, and overall ranking performance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。