用信息熵指导搜索，自动设计高性能稠密网络。
Dense Optimizer : An Information Entropy-Guided Structural Search Method for Dense-like Neural Network Design
- 将稠密网络视为分层信息系统，以熵最大化为优化目标
- 4小时完成搜索，CIFAR-100上准确率提升5.97%至84.3%
- 适合追求自动架构设计与高精度模型的研究者
稠密卷积网络因其轻量高效结构被持续优化，但现有稠密类架构主要依赖人工设计，难以基于经验调整通道数与特征复用程度。为此，本文提出一种名为Dense Optimizer的自动架构搜索方法。将稠密网络视为分层信息系统，通过最大化网络信息熵并约束各阶段熵分布遵循幂律，构建优化问题。提出分支定界算法，紧密融合幂律原则与搜索空间缩放，高效求解。在多个视觉基准数据集上验证其优越性：仅用一台CPU运行4小时即可完成高质量搜索。所搜模型DenseNet-OPT在CIFAR-100上达到84.3%的顶级准确率，比原版高出5.97%。
原文摘要 · Abstract (English)
Dense Convolutional Network has been continuously refined to adopt a highly efficient and compact architecture, owing to its lightweight and efficient structure. However, the current Dense-like architectures are mainly designed manually, it becomes increasingly difficult to adjust the channels and reuse level based on past experience. As such, we propose an architecture search method called Dense Optimizer that can search high-performance dense-like network automatically. In Dense Optimizer, we view the dense network as a hierarchical information system, maximize the network's information entropy while constraining the distribution of the entropy across each stage via a power law, thereby constructing an optimization problem. We also propose a branch-and-bound optimization algorithm, tightly integrates power-law principle with search space scaling to solve the optimization problem efficiently. The superiority of Dense Optimizer has been validated on different computer vision benchmark datasets. Specifically, Dense Optimizer completes high-quality search but only costs 4 hours with one CPU. Our searched model DenseNet-OPT achieved a top 1 accuracy of 84.3% on CIFAR-100, which is 5.97% higher than the original one.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。