自适应调整丢弃策略，提升深度网络训练效果。
Adaptive Tabu Dropout for Regularization of Deep Neural Network
- 引入可变保留周期，让被丢弃单元在不同周期内不被重复丢弃。
- 实验表明，新方法在多个基准数据集上显著优于标准丢弃法。
- 自动调节保留周期，适合追求模型泛化性能的研究者。
Dropout 是一种有效的深度神经网络正则化策略。通过将最近一轮中被丢弃的神经元设为‘禁用’状态（tabu），并保持其参与后续训练，可实现丢弃策略的多样性。本文针对该机制提出两项改进：首先，引入‘保留周期’（tabu tenure），即特定神经元在多少轮内不会被丢弃，不同周期设置可依据搜索空间特性增强训练多样性；其次，提出自适应算法，根据训练过程中的性能变化动态选择保留周期。在多个标准基准数据集上的实验结果表明，自适应表禁忌丢弃与周期化表禁忌丢弃均显著优于标准丢弃和基础表禁忌丢弃机制。
原文摘要 · Abstract (English)
Dropout is an effective strategy for the regularization of deep neural networks. Applying tabu to the units that have been dropped in the recent epoch and retaining them for training ensures diversification in dropout. In this paper, we improve the Tabu Dropout mechanism for training deep neural networks in two ways. Firstly, we propose to use tabu tenure, or the number of epochs a particular unit will not be dropped. Different tabu tenures provide diversification to boost the training of deep neural networks based on the search landscape. Secondly, we propose an adaptive tabu algorithm that automatically selects the tabu tenure based on the training performances through epochs. On several standard benchmark datasets, the experimental results show that the adaptive tabu dropout and tabu tenure dropout diversify and perform significantly better compared to the standard dropout and basic tabu dropout mechanisms.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。