用随机平均池化替代传统池化，提升模型泛化能力且无性能损失。
Stochastic Subsampling With Average Pooling
- 在池化层引入类似Dropout的随机性，保持输出一致性
- 替换现有平均池化后，在多任务、多数据集上均实现稳定提升
- 无需修改网络结构，可直接嵌入现有深度模型
深度神经网络的正则化是提升泛化性能、避免过拟合的关键。尽管流行的Dropout方法具有正则化效果，但会导致输出不一致，可能损害模型性能。本文提出一种新模块——随机平均池化（stochastic average pooling），在池化操作中引入类似Dropout的随机性，同时保留平均池化的稳定性。我们分析了随机下采样与平均池化的特性，并设计出无输出不一致问题的模块。该方法在不引发性能下降的前提下实现正则化效果，可无缝集成至现有深度网络架构。实验表明，将原有平均池化替换为随机平均池化，在多种任务、数据集和模型上均带来一致性的性能提升。
原文摘要 · Abstract (English)
Regularization of deep neural networks has been an important issue to achieve higher generalization performance without overfitting problems. Although the popular method of Dropout provides a regularization effect, it causes inconsistent properties in the output, which may degrade the performance of deep neural networks. In this study, we propose a new module called stochastic average pooling, which incorporates Dropout-like stochasticity in pooling. We describe the properties of stochastic subsampling and average pooling and leverage them to design a module without any inconsistency problem. The stochastic average pooling achieves a regularization effect without any potential performance degradation due to the inconsistency issue and can easily be plugged into existing architectures of deep neural networks. Experiments demonstrate that replacing existing average pooling with stochastic average pooling yields consistent improvements across a variety of tasks, datasets, and models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。