首个面向模拟存内计算的神经架构搜索基准，揭示高效架构设计规律。
AnalogNAS-Bench: A NAS Benchmark for Analog In-Memory Computing
- 构建首个专为模拟存内计算设计的NAS基准，融合硬件非理想特性。
- 发现宽而分支的模块更抗噪声，跳跃连接可缓解时序漂移。
- 适合研究模拟存内计算与神经架构搜索交叉方向的学者。
模拟存内计算（AIMC）作为加速深度神经网络的高效范式，相比传统数字硬件具有显著的能效和延迟优势。然而，现有先进神经网络未针对AIMC特性设计，难以应对其独特的非理想性。因此需要神经架构搜索（NAS）系统性地发现专为AIMC约束优化的网络结构。但现有方法缺乏考虑硬件非理想性的专用基准，制约了对比与洞察。为此，本文提出首个面向AIMC的NAS基准——AnalogNAS-Bench。研究揭示三大关键发现：(1) 标准量化技术无法捕捉AIMC特有的噪声；(2) 鲁棒架构倾向于采用更宽、分支更多的模块；(3) 跳跃连接有助于提升对时序漂移噪声的鲁棒性。这些发现揭示了当前基准的局限性，为未来模拟感知的NAS研究铺平道路。所有代码实现详见https://github.com/IBM/analog-nas/tree/main/analognasbench。
原文摘要 · Abstract (English)
Analog In-memory Computing (AIMC) has emerged as a highly efficient paradigm for accelerating Deep Neural Networks (DNNs), offering significant energy and latency benefits over conventional digital hardware. However, state-of-the-art neural networks are not inherently designed for AIMC, as they fail to account for its unique non-idealities. Neural Architecture Search (NAS) is thus needed to systematically discover neural architectures optimized explicitly for AIMC constraints. However, comparing NAS methodologies and extracting insights about robust architectures for AIMC requires a dedicated NAS benchmark that explicitly accounts for AIMC-specific hardware non-idealities. To address this, we introduce AnalogNAS-Bench, the first NAS benchmark tailored specifically for AIMC. Our study reveals three key insights: (1) standard quantization techniques fail to capture AIMC-specific noises, (2) robust architectures tend to feature wider and branched blocks, (3) skip connections improve resilience to temporal drift noise. These insights highlight the limitations of current NAS benchmarks for AIMC and pave the way for future analog-aware NAS. All the implementations used in this paper can be found at https://github.com/IBM/analog-nas/tree/main/analognasbench.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。