用合成数据和隐藏激活快速训练神经网络，精度高且只需少量参数。
Prototype Training with Dual Pseudo-Inverse and Optimized Hidden Activations
- 通过闭式解更新权重，仅优化合成原型和隐藏激活。
- 在MNIST和Fashion-MNIST上分别达97.8%和89.3%准确率，训练仅需4.5秒。
- 适合追求高速低参模型的开发者，支持多层优化与理论分析。
我们提出Proto-PINV+H，一种快速训练范式，结合闭式权重计算与小规模合成输入、软标签及关键隐藏激活的梯度优化。每轮迭代中，通过两次（或更多）岭正则化伪逆求解闭式重算所有权重矩阵，仅用Adam更新原型。可训练自由度从权重空间转移至数据/激活空间。在MNIST（60k训练，10k测试）和Fashion-MNIST（60k训练，10k测试）上，分别实现97.8%和89.3%的测试准确率，使用约13万可训练参数、250轮迭代，在RTX 5060（16GB）上耗时3.9–4.5秒。方法支持多层扩展（每层优化隐藏激活）、可学习岭参数、可选PCA/PLS投影，并提供原型矩阵条件数与泛化能力的理论关联。相较ELM、随机特征岭回归及反向传播训练的浅层MLP，该方法在准确率-速度-规模间表现更优。
原文摘要 · Abstract (English)
We present Proto-PINV+H, a fast training paradigm that combines closed-form weight computation with gradient-based optimisation of a small set of synthetic inputs, soft labels, and-crucially-hidden activations. At each iteration we recompute all weight matrices in closed form via two (or more) ridge-regularised pseudo-inverse solves, while updating only the prototypes with Adam. The trainable degrees of freedom are thus shifted from weight space to data/activation space. On MNIST (60k train, 10k test) and Fashion-MNIST (60k train, 10k test), our method reaches 97.8% and 89.3% test accuracy on the official 10k test sets, respectively, in 3.9s--4.5s using approximately 130k trainable parameters and only 250 epochs on an RTX 5060 (16GB). We provide a multi-layer extension (optimised activations at each hidden stage), learnable ridge parameters, optional PCA/PLS projections, and theory linking the condition number of prototype matrices to generalisation. The approach yields favourable accuracy--speed--size trade-offs against ELM, random-feature ridge, and shallow MLPs trained by back-propagation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。