对比遗传算法与梯度下降在小样本医学数据上的神经网络训练效果
Genetic algorithm vs. gradient descent for training a neural network architecture dedicated to low data regimes in small medical datasets
- 为基于空间距离的神经网络设计专用反向传播，尝试用梯度下降替代遗传算法
- 在4个医疗数据集上，遗传算法分类准确率均显著高于梯度下降（最高差17%）
- 适合小样本、参数高度耦合的生物启发式神经网络，不推荐传统梯度优化
本研究旨在设计并实现一种针对距离编码生物信息神经网络（DEBI-NN）的梯度下降学习器，并与传统的遗传算法（GA）进行性能比较。DEBI-NN通过神经元在欧几里得空间中的位置距离定义连接权重，极大减少可训练参数。研究采用定制化空间反向传播方案，在合成非线性“两月”数据集及三个临床医学影像组学数据集（DLBCL, HECKTOR）和胎儿心电图数据集上，样本量介于85至2126之间，对两种优化器进行调参对比。结果显示，所有实验中遗传算法均表现更优：合成数据集100% vs 83%，DLBCL 83% vs 78%，HECKTOR 80% vs 67%，胎儿数据集81% vs 66%；而梯度下降因神经元间依赖导致梯度纠缠，无法有效捕捉空间编码的非线性特征，表现出不稳定性。结果表明，基于梯度的方法在高度空间耦合的架构中存在根本局限，验证了进化策略在训练DEBI-NN中的适用性。
原文摘要 · Abstract (English)
Aim/Introduction: Distance-encoding biomorphic-informational neural network (DEBI-NN) is a recently proposed architecture in which connection weights are defined by the distances between neurons positioned in a Euclidian space. This approach drastically reduces the number of trainable parameters compared to classical neural networks in which weights are directly trained. The training process for DEBI-NN is based on a genetic algorithm (GA), rather than gradient descent (GD) which remains the prevailing optimization algorithm in deep learning. We aim to design and implement a GD learner for DEBI-NN and assess its performance compared to GA. Materials and Methods: We designed a spatial backpropagation scheme tailored to DEBI-NN and carried out a comparison between GD and GA for classification tasks, using a synthetic non-linear "two-moons" dataset, two clinical medical imaging radiomic datasets and a fetal cardiotocography dataset with a sample sizes ranging from n=85 to n=2126. Each optimizer was tuned through targeted hyperparameter searches adapted to each dataset. Results: Across all experiments, GA consistently produced superior decision boundaries and classification performance (Synthetic: 100% vs 83%; DLBCL: 83% vs 78%; HECKTOR: 80% vs 67%; Fetal: 81% vs 66%), whereas GD exhibited instability and failed to fully capture the non-linear patterns inherent to DEBI-NN's spatial encoding. The entangled gradients resulting from neuron interdependencies limit the effectiveness of classical backpropagation. Conclusion: These findings highlight fundamental limitations of gradient-based methods in architectures with highly interdependent spatial parameters and confirm the suitability of evolutionary strategies for training DEBI-NN.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。