用进化算法自动设计适配任务的激活函数，提升模型性能且不增加参数量。
Task-Specific Activation Functions for Neuroevolution using Grammatical Evolution
- 通过语法进化自动演化专用激活函数，无需人工设计。
- 在多个二分类数据集上F1分数提升2.4%至9.4%，与ReLU相同架构下对比。
- 适合资源受限设备，兼顾高效训练与推理，适合轻量化部署场景。
激活函数对神经网络的性能和行为至关重要，显著影响其学习与泛化能力。传统激活函数如ReLU、Sigmoid和Tanh虽广泛应用，但未必适用于所有任务与数据集。本文提出Neuvo GEAF方法，利用语法进化（GE）自动演化适配特定网络架构与数据集的新激活函数。在多个知名二分类数据集上的实验表明，使用相同网络架构时，新函数相比ReLU在F1分数上实现2.4%至9.4%的统计显著提升。这些性能增益在不增加参数量的前提下达成，支持向更高效神经网络发展的趋势，使其能在资源受限的边缘设备上有效运行。结果表明，演化得到的激活函数可显著提升紧凑型网络性能，同时保持训练与推理阶段的能效优势。
原文摘要 · Abstract (English)
Activation functions play a critical role in the performance and behaviour of neural networks, significantly impacting their ability to learn and generalise. Traditional activation functions, such as ReLU, sigmoid, and tanh, have been widely used with considerable success. However, these functions may not always provide optimal performance for all tasks and datasets. In this paper, we introduce Neuvo GEAF - an innovative approach leveraging grammatical evolution (GE) to automatically evolve novel activation functions tailored to specific neural network architectures and datasets. Experiments conducted on well-known binary classification datasets show statistically significant improvements in F1-score (between 2.4% and 9.4%) over ReLU using identical network architectures. Notably, these performance gains were achieved without increasing the network's parameter count, supporting the trend toward more efficient neural networks that can operate effectively on resource-constrained edge devices. This paper's findings suggest that evolved activation functions can provide significant performance improvements for compact networks while maintaining energy efficiency during both training and inference phases.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。