ULU激活函数分段处理正负输入，性能优于ReLU和Mish。
ULU: A Unified Activation Function
- 设计分段非单调激活函数，正负输入用不同参数处理。
- 在图像分类和目标检测任务中显著超越ReLU与Mish。
- 提出可学习变体AULU，适合需要自适应响应的模型设计。
我们提出一种新型非单调分段激活函数ULU，定义为：当x<0时使用f(x;α₁)，当x≥0时使用f(x;α₂)，其中f(x;α)=0.5x(tanh(αx)+1)，α>0。ULU对正负输入采用差异化处理。大量实验表明，ULU在图像分类和目标检测任务中显著优于ReLU和Mish。其变体AULU将参数设为可学习的β₁²和β₂²，使模型能分别自适应正负输入响应。此外，我们从AULU引入了LIB（Like Inductive Bias）指标，用于定量衡量模型的归纳偏置。
原文摘要 · Abstract (English)
We propose \textbf{ULU}, a novel non-monotonic, piecewise activation function defined as $\{f(x;α_1),x<0; f(x;α_2),x>=0 \}$, where $f(x;α)=0.5x(tanh(αx)+1),α>0$. ULU treats positive and negative inputs differently. Extensive experiments demonstrate ULU significantly outperforms ReLU and Mish across image classification and object detection tasks. Its variant Adaptive ULU (\textbf{AULU}) is expressed as $\{f(x;β_1^2),x<0; f(x;β_2^2),x>=0 \}$, where $β_1$ and $β_2$ are learnable parameters, enabling it to adapt its response separately for positive and negative inputs. Additionally, we introduce the LIB (Like Inductive Bias) metric from AULU to quantitatively measure the inductive bias of the model.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。