通过降低卷积特征图相似性，提升CNN模型效率与性能。
Feature Map Similarity Reduction in Convolutional Neural Networks
- 提出卷积相似性最小化方法，独立于输入数据优化特征图
- 减少冗余后准确率提升且收敛速度加快，小模型也能达同等效果
- 适合追求轻量化部署或高效训练的深度学习研究者
卷积神经网络（CNN）存在特征图冗余问题，导致模型容量利用效率低。现有工作多聚焦于核正交化方法，但本文从理论和实证上证明：核正交化并不必然减少特征图冗余。基于此，我们提出卷积相似性（Convolutional Similarity）方法，用于独立于输入数据地降低特征图相似性。该方法可作为正则项或迭代初始化策略进行最小化。实验表明，最小化卷积相似性不仅提升了分类准确率，还加速了收敛过程。此外，该方法使更小的模型即可达到相同性能水平，显著提升模型容量利用率。未来工作将探索该初始化方法与优化动量结合的可能性，并考察其在生成框架中的影响。
原文摘要 · Abstract (English)
It has been observed that Convolutional Neural Networks (CNNs) suffer from redundancy in feature maps, leading to inefficient capacity utilization. Efforts to address this issue have largely focused on kernel orthogonality method. In this work, we theoretically and empirically demonstrate that kernel orthogonality does not necessarily lead to a reduction in feature map redundancy. Based on this analysis, we propose the Convolutional Similarity method to reduce feature map similarity, independently of the CNN's input. The Convolutional Similarity can be minimized as either a regularization term or an iterative initialization method. Experimental results show that minimizing Convolutional Similarity not only improves classification accuracy but also accelerates convergence. Furthermore, our method enables the use of significantly smaller models to achieve the same level of performance, promoting a more efficient use of model capacity. Future work will focus on coupling the iterative initialization method with the optimization momentum term and examining the method's impact on generative frameworks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。