改进随机配置网络的评估机制,提升学习效率与可扩展性
Deeper Insights into Learning Performance of Stochastic Configuration Networks
- 用新监督机制直接评估随机基函数的误差下降潜力
- 无需计算广义逆矩阵,降低计算复杂度,提升训练速度
- 在基准数据集上显著优于传统SCN,适合大规模建模
随机配置网络(SCNs)是一类将随机算法融入增量学习框架的随机神经网络。其核心特征是监督机制,可自适应调整分布以生成有效的随机基函数,实现无误差学习。本文全面分析了监督机制对SCN学习性能的影响。研究发现,当前框架通过误差下降潜力的下界评估每个基函数的有效性,限制了整体学习效率,可能导致每轮迭代中无法持续选择最优候选基函数。为此,我们提出一种新方法,通过分析隐藏层输出矩阵来评估误差下降潜力,无需计算输出矩阵的Moore-Penrose逆。该方法提升了基函数选择精度,降低了计算复杂度,增强了可扩展性与学习能力。基于此提出递归广义逆-随机配置网络(RMPI-SCN)训练方案,并在多个基准数据集上通过仿真验证其有效性。实验表明,RMPI-SCN在学习能力上显著优于传统SCN,展现出在大规模数据建模中的应用潜力。
原文摘要 · Abstract (English)
Stochastic Configuration Networks (SCNs) are a class of randomized neural networks that integrate randomized algorithms within an incremental learning framework. A defining feature of SCNs is the supervisory mechanism, which adaptively adjusts the distribution to generate effective random basis functions, thereby enabling error-free learning. In this paper, we present a comprehensive analysis of the impact of the supervisory mechanism on the learning performance of SCNs. Our findings reveal that the current SCN framework evaluates the effectiveness of each random basis function in reducing residual errors using a lower bound on its error reduction potential, which constrains SCNs' overall learning efficiency. Specifically, SCNs may fail to consistently select the most effective random candidate as the new basis function during each training iteration. To overcome this problem, we propose a novel method for evaluating the hidden layer's output matrix, supported by a new supervisory mechanism that accurately assesses the error reduction potential of random basis functions without requiring the computation of the Moore-Penrose inverse of the output matrix. This approach enhances the selection of basis functions, reducing computational complexity and improving the overall scalability and learning capabilities of SCNs. We introduce a Recursive Moore-Penrose Inverse-SCN (RMPI-SCN) training scheme based on the new supervisory mechanism and demonstrate its effectiveness through simulations over some benchmark datasets. Experiments show that RMPI-SCN outperforms the conventional SCN in terms of learning capability, underscoring its potential to advance the SCN framework for large-scale data modeling applications.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。