用字符串嵌入预测模型性能，让硬件感知的神经网络搜索更灵活。
SEval-NAS: A Search-Agnostic Evaluation for Neural Architecture Search
- 将网络结构转为字符串并向量化，预测精度、延迟和内存消耗。
- 在NATS-Bench和HW-NAS-Bench上对延迟和内存预测效果优于精度。
- 可无缝接入现有搜索框架，适合想扩展评估指标的研究者。
神经网络架构搜索（NAS）自动化发现满足特定条件的网络结构，但其评估流程常被硬编码，难以引入新指标。这一问题在硬件感知的NAS中尤为突出，因目标设备（如边缘设备）决定了优化目标。为此，我们提出SEval-NAS，一种与搜索无关的评估机制：将架构转换为字符串，进行向量嵌入，并预测性能指标。在NATS-Bench和HW-NAS-Bench数据集上，评估了准确率、延迟和内存占用。肯德尔τ相关系数显示，该方法对延迟和内存的预测能力优于准确率，表明其作为硬件成本预测器的适用性。进一步将SEval-NAS集成到FreeREA中，成功对原生不支持的指标进行评估，保持了搜索时间，仅需极少算法修改。代码已开源：https://github.com/Analytics-Everywhere-Lab/neural-architecture-search
原文摘要 · Abstract (English)
Neural architecture search (NAS) automates the discovery of neural networks that meet specified criteria, yet its evaluation procedures are often hardcoded, limiting the ability to introduce new metrics. This issue is especially pronounced in hardware-aware NAS, where objectives depend on target devices such as edge hardware. To address this limitation, we propose SEval-NAS, a metric-evaluation mechanism that converts architectures to strings, embeds them as vectors, and predicts performance metrics. Using NATS-Bench and HW-NAS-Bench, we evaluated accuracy, latency, and memory. Kendall's $τ$ correlations showed stronger latency and memory predictions than accuracy, indicating the suitability of SEval-NAS as a hardware cost predictor. We further integrated SEval-NAS into FreeREA to evaluate metrics not originally included. The method successfully ranked FreeREA-generated architectures, maintained search time, and required minimal algorithmic changes. Our implementation is available at: https://github.com/Analytics-Everywhere-Lab/neural-architecture-search
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。