用高斯-牛顿近似加速神经网络生长，更高效找到最优扩展方式。
Learning Morphisms with Gauss-Newton Approximation for Growing Networks
- 通过损失函数的高斯-牛顿近似评估网络变形操作
- 在CIFAR-10/100上达到相近或更优性能，计算成本更低
- 适合追求高效架构搜索的实践者和研究者
神经架构搜索（NAS）中一种流行方法是通过局部小改动（即网络形态变换）逐步增长网络。这类方法从一个小型初始网络开始，自动添加新神经元以扩展结构。然而，如何高效确定网络中最佳扩展位置仍具挑战。本文提出一种基于损失函数高斯-牛顿近似的NAS方法，用于高效学习和评估候选网络形态变换。我们在CIFAR-10和CIFAR-100分类任务上与当前最优的NAS方法进行比较，结果表明该方法在更小的计算开销下，能够学习到质量相当或更优的网络架构。
原文摘要 · Abstract (English)
A popular method for Neural Architecture Search (NAS) is based on growing networks via small local changes to the network's architecture called network morphisms. These methods start with a small seed network and progressively grow the network by adding new neurons in an automated way. However, it remains a challenge to efficiently determine which parts of the network are best to grow. Here we propose a NAS method for growing a network by using a Gauss-Newton approximation of the loss function to efficiently learn and evaluate candidate network morphisms. We compare our method with state of the art NAS methods for CIFAR-10 and CIFAR-100 classification tasks, and conclude our method learns similar quality or better architectures at a smaller computational cost.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。