用遗传算法找高效神经网络中的彩票子网，不需训练也能高精度。
Towards Scalable Lottery Ticket Networks using Genetic Algorithms
- 用遗传算法搜索初始网络中的高精度子网，无需梯度信息。
- 在分类任务中达到比现有方法更高的准确率和稀疏度。
- 适合追求低资源消耗、无需训练的模型部署场景。
构建既高效又有效的现代深度学习系统，需重新思考模型训练与神经架构设计的既有范式。与其先训练高度过参数化的网络再压缩，不如直接设计高性能、低参数量的网络。强彩票票券假说指出，在随机初始化且充分过参数化的神经网络中，存在无需训练即可达到原模型精度的子网络。本文探索使用遗传算法识别这些强彩票票券子网络。实验表明，在二分类与多分类任务中，该方法在不依赖梯度信息的情况下，取得了优于当前最优水平的准确率与稀疏度。此外，本文还强调了在扩展至更复杂网络架构与学习任务时，需采用合适的评估指标。
原文摘要 · Abstract (English)
Building modern deep learning systems that are not just effective but also efficient requires rethinking established paradigms for model training and neural architecture design. Instead of adapting highly overparameterized networks and subsequently applying model compression techniques to reduce resource consumption, a new class of high-performing networks skips the need for expensive parameter updates, while requiring only a fraction of parameters, making them highly scalable. The Strong Lottery Ticket Hypothesis posits that within randomly initialized, sufficiently overparameterized neural networks, there exist subnetworks that can match the accuracy of the trained original model-without any training. This work explores the usage of genetic algorithms for identifying these strong lottery ticket subnetworks. We find that for instances of binary and multi-class classification tasks, our approach achieves better accuracies and sparsity levels than the current state-of-the-art without requiring any gradient information. In addition, we provide justification for the need for appropriate evaluation metrics when scaling to more complex network architectures and learning tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。