揭示隐层奇异值结构如何决定ELM的数值稳定性
Spectral Stability of Pseudoinverse-Based Extreme Learning Machine
- 从谱分析视角研究基于伪逆的ELM，发现最小奇异值决定输出权重扰动放大程度
- 条件数量化隐层不稳定性，在病态条件下SVD方法比迭代法更可靠
- 适用于关注ELM训练稳定性的机器学习研究者与工程应用开发者
极限学习机（ELM）通过摩尔-彭罗斯伪逆解析计算输出权重，虽训练快速，但其数值稳定性高度依赖于隐层矩阵的条件性。本文从谱角度研究基于伪逆的ELM，表明最小奇异值决定了输出权重的扰动放大效应，而条件数则提供了隐层不稳定的定量度量。对比了基于SVD的伪逆计算与迭代超幂方法，通过随机特征解释讨论了宽度相关的条件性。在合成矩阵和ELM基准测试上的实验显示，当条件不良时，SVD方法仍保持最可靠性，而迭代方法对谱特性更敏感。结果表明，ELM的稳定性根本上由隐层矩阵的奇异值结构决定。
原文摘要 · Abstract (English)
Extreme Learning Machine (ELM) computes output weights analytically using the Moore-Penrose pseudoinverse. Although this leads to fast training, its numerical stability depends strongly on the conditioning of the hidden layer matrix. This paper studies pseudoinverse-based ELM from a spectral perspective. We show that the smallest singular value governs perturbation amplification in the output weights, while the condition number provides a quantitative measure of hidden-layer instability. We compare SVD-based pseudoinverse computation with iterative hyperpower methods and discuss width-dependent conditioning through a random feature interpretation. Experiments on synthetic matrices and ELM benchmarks show that SVD-based methods remain the most reliable under ill conditioning, while iterative methods are more sensitive to spectral properties. The results suggest that ELM stability is fundamentally governed by the singular value structure of the hidden layer matrix.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。