不用激活函数的图像识别模型，用多项式替代非线性，性能更优。
Activation-Free Backbones for Image Recognition: Polynomial Alternatives within MetaFormer-Style Vision Models
- 用哈达玛积替代激活函数，构建无激活的多项式模块
- 在ImageNet等任务上达到或超越传统模型性能
- 计算成本更低，适合追求高效推理的场景
现代视觉骨干网络依赖点乘激活函数（如ReLU、GELU）和指数Softmax作为非线性来源，但我们证明在MetaFormer风格的视觉模型中，这些并非必需。本文设计了三种核心组件（MLP、卷积、注意力）的无激活多项式替代方案，以哈达玛积取代标准非线性，生成输入的多项式函数。这些模块可无缝集成至现有架构：在MetaFormer框架内实现的PolyNeXt模型，在ImageNet分类、ADE20K语义分割及分布外鲁棒性任务上，表现与或优于基于激活的对应模型，且在降低计算成本的同时显著超越先前多项式网络，表明标准模块的多项式变体可优于复杂定制架构。
原文摘要 · Abstract (English)
Modern vision backbones treat pointwise activations (e.g., ReLU, GELU) and exponential softmax as essential sources of nonlinearity, but we demonstrate they are not required within MetaFormer-style vision backbones. We design activation-free polynomial alternatives for three core primitives (MLPs, convolutions, and attention), where Hadamard products replace standard nonlinearities to yield polynomial functions of the input. These modules integrate seamlessly into existing architectures: instantiated within MetaFormer, a modular framework for vision backbones, our PolyNeXt models match or exceed activation-based counterparts across model scales on ImageNet classification, ADE20K semantic segmentation, and out-of-distribution robustness. We also substantially outperform prior polynomial networks at reduced computational cost, showing that polynomial variants of standard modules beat complex custom architectures.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。