通过谱特性搜索神经网络架构，实现更少参数的高效模型设计。
Spectral Architecture Search for Neural Network Models
- 利用层间传输矩阵的谱特性构建可微连续空间
- 自动生成表达能力最低但任务适配的精简架构
- 适合追求轻量化模型与可优化架构搜索的研究者
神经网络的架构设计与优化是该领域的重要挑战。本文提出SPARCS（SPectral ARchiteCture Search），一种基于层间传输矩阵谱特性的新型架构搜索方法。该方法通过构建连续且可微的流形空间，使梯度优化算法得以应用。在简单基准模型上验证，新方法能生成表达能力最小化且参数量更低的自适应架构，优于其他可行方案。
原文摘要 · Abstract (English)
Architecture design and optimization are challenging problems in the field of artificial neural networks. Working in this context, we here present SPARCS (SPectral ARchiteCture Search), a novel architecture search protocol which exploits the spectral attributes of the inter-layer transfer matrices. SPARCS allows one to explore the space of possible architectures by spanning continuous and differentiable manifolds, thus enabling for gradient-based optimization algorithms to be eventually employed. With reference to simple benchmark models, we show that the newly proposed method yields a self-emerging architecture with a minimal degree of expressivity to handle the task under investigation and with a reduced parameter count as compared to other viable alternatives.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。