arXiv:2410.01189cs.CVcs.DL2024-10

复现并验证了网络去卷积提升模型性能的结论。

[Re] Network Deconvolution

  • 用网络去卷积替代批归一化,消除层间相关性以提升训练效果。
  • 367次实验中,14项结果完全复现,准确率偏差小于10%。
  • 适合关注模型优化与可复现性研究的深度学习从业者。

本研究旨在复现Ye等人于2020年发表的《网络去卷积》中的成果。该方法提出一种优化技术,用于卷积神经网络训练,通过在数据输入各层前去除像素级和通道级相关性来提升模型性能。我们重点验证了该技术替代批归一化可提高模型表现的主张。经367次跨架构、数据集及超参数配置的实验,成功复现了原文表1与表2的结果:表1中准确率存在小于10%的微小偏差,整体趋势一致;表2全部14项结果均与原论文一致。同时,我们记录了在CIFAR-10/CIFAR-100上使用1、20、100个周期训练时各架构的训练与测试时间,并记录了表2在ImageNet上的总执行时间。所有数据与代码已公开于https://github.com/lamps-lab/rep-network-deconvolution。

原文摘要 · Abstract (English)

Our work aims to reproduce the set of findings published in "Network Deconvolution" by Ye et al. (2020)[1]. That paper proposes an optimization technique for model training in convolutional neural networks. The proposed technique "network deconvolution" is used in convolutional neural networks to remove pixel-wise and channel-wise correlations before data is fed into each layer. In particular, we interrogate the validity of the authors' claim that using network deconvolution instead of batch normalization improves deep learning model performance. Our effort confirms the validity of this claim, successfully reproducing the results reported in Tables 1 and 2 of the original paper. Our study involved 367 unique experiments across multiple architectures, datasets, and hyper parameter configurations. For Table 1, while there were some minor deviations in accuracy when compared to the original values (within 10%), the overall trend was consistent with the original study's findings when training the models with epochs 20 and 100. For Table 2, all 14 reproduced values were consistent with the original values. Additionally, we document the training and testing times for each architecture in Table 1 with 1, 20, and 100 epoch settings for both CIFAR-10 and CIFAR-100 datasets. We document the total execution times for Table 2 architectures with the ImageNet dataset. The data and software used for this reproducibility study are publicly available at https://github.com/lamps-lab/rep-network-deconvolution.

深度学习可复现性模型优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。