动态扩展任意有向无环图神经网络，高效节省计算资源。
Growth strategies for arbitrary DAG neural architectures
- 基于反向传播信息，训练中自由生长任意拓扑结构的神经网络。
- 通过消除表达瓶颈，减少冗余计算，提升参数效率。
- 适合需要动态扩容且关注推理效率的研究者与工程师。
深度学习虽取得显著成果，但大规模模型带来高昂的训练与推理成本。本文旨在降低训练和推理时间，聚焦神经架构生长技术：在训练过程中根据反向传播信息，按需动态扩展小型模型规模。我们突破现有方法限制，支持任意有向无环图（DAG）结构的自由增长，并通过缓解表达能力瓶颈，减少过度计算，引导网络向更高效的参数配置演化。
原文摘要 · Abstract (English)
Deep learning has shown impressive results obtained at the cost of training huge neural networks. However, the larger the architecture, the higher the computational, financial, and environmental costs during training and inference. We aim at reducing both training and inference durations. We focus on Neural Architecture Growth, which can increase the size of a small model when needed, directly during training using information from the backpropagation. We expand existing work and freely grow neural networks in the form of any Directed Acyclic Graph by reducing expressivity bottlenecks in the architecture. We explore strategies to reduce excessive computations and steer network growth toward more parameter-efficient architectures.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。