将NEAT神经进化网络改造为可高效GPU加速的反向传播结构
PropNEAT -- Efficient GPU-Compatible Backpropagation over NeuroEvolutionary Augmenting Topology Networks
- 通过基因图到层结构的双向映射，保留NEAT拓扑同时支持GPU反向传播
- 在58个数据集上表现仅次于随机森林，训练速度远超原始NEAT
- 适合需要轻量化模型的低功耗场景，可拓展至强化学习与卷积网络
我们提出PropNEAT，一种高效的神经进化增强拓扑网络（NEAT）反向传播实现。该方法通过基因图到分层架构的双向映射，在保持原始NEAT基因组结构的同时，支持高效的GPU反向传播。我们在宾夕法尼亚机器学习基准测试数据库中的58个二分类数据集上进行了测试，对比了逻辑回归、全连接神经网络、随机森林以及重新训练的PropNEAT模型。结果显示，PropNEAT整体性能仅次于随机森林，但与其他模型差异仅在随机森林与逻辑回归及重训模型之间具有统计显著性。相比朴素反向传播方法，PropNEAT显著更快；两者均远快于原始NEAT。我们验证了PropNEAT每轮训练时间随网络深度线性增长，且在GPU上高效运行。该实现可扩展至强化学习或卷积网络，并能发现更稀疏、更小型的网络，适用于低功耗应用场景。
原文摘要 · Abstract (English)
We introduce PropNEAT, a fast backpropagation implementation of NEAT that uses a bidirectional mapping of the genome graph to a layer-based architecture that preserves the NEAT genomes whilst enabling efficient GPU backpropagation. We test PropNEAT on 58 binary classification datasets from the Penn Machine Learning Benchmarks database, comparing the performance against logistic regression, dense neural networks and random forests, as well as a densely retrained variant of the final PropNEAT model. PropNEAT had the second best overall performance, behind Random Forest, though the difference between the models was not statistically significant apart from between Random Forest in comparison with logistic regression and the PropNEAT retrain models. PropNEAT was substantially faster than a naive backpropagation method, and both were substantially faster and had better performance than the original NEAT implementation. We demonstrate that the per-epoch training time for PropNEAT scales linearly with network depth, and is efficient on GPU implementations for backpropagation. This implementation could be extended to support reinforcement learning or convolutional networks, and is able to find sparser and smaller networks with potential for applications in low-power contexts.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。