提出M因子评估指标,平衡模型精度与大小,适配移动端等资源受限场景。
The M-factor: A Novel Metric for Evaluating Neural Architecture Search in Resource-Constrained Environments
- 引入M因子综合衡量精度与模型大小,替代单一性能指标。
- 在19,683种ResNet配置中,强化学习法达0.84的最高M因子值。
- 揭示随机搜索在某些情况下可媲美复杂算法,指导高效NAS策略选择。
神经架构搜索(NAS)旨在自动化设计深度神经网络,但现有方法多聚焦于最大化精度,忽视模型效率,限制了其在移动设备和边缘计算等资源受限环境中的应用。当前评估指标也偏重性能而缺乏对效率的平衡考量。为此,本文提出M因子这一新指标,融合模型精度与大小,用于评估适合受限场景的架构。研究对比了四种不同范式的NAS方法:基于策略的强化学习、正则化进化、树结构帕尔森估计器(TPE)和多轮随机搜索。在CIFAR-10数据集上,针对包含19,683种配置的ResNet搜索空间进行实验。结果显示,基于策略的强化学习与正则化进化分别获得0.84和0.82的M因子,多轮随机搜索为0.75,TPE为0.67。前者在39次试验后表现稳定,后者在20次内完成优化。研究分析了各方法的优化动态与精度-尺寸权衡特性,发现随机搜索在使用M因子评估时表现不逊于复杂算法。结果表明,M因子有效弥补了传统指标缺陷,引导NAS向兼顾性能与效率的方向发展,为资源受限场景下的策略选择提供重要参考。
原文摘要 · Abstract (English)
Neural Architecture Search (NAS) aims to automate the design of deep neural networks. However, existing NAS techniques often focus on maximising accuracy, neglecting model efficiency. This limitation restricts their use in resource-constrained environments like mobile devices and edge computing systems. Moreover, current evaluation metrics prioritise performance over efficiency, lacking a balanced approach for assessing architectures suitable for constrained scenarios. To address these challenges, this paper introduces the M-factor, a novel metric combining model accuracy and size. Four diverse NAS techniques are compared: Policy-Based Reinforcement Learning, Regularised Evolution, Tree-structured Parzen Estimator (TPE), and Multi-trial Random Search. These techniques represent different NAS paradigms, providing a comprehensive evaluation of the M-factor. The study analyses ResNet configurations on the CIFAR-10 dataset, with a search space of 19,683 configurations. Experiments reveal that Policy-Based Reinforcement Learning and Regularised Evolution achieved M-factor values of 0.84 and 0.82, respectively, while Multi-trial Random Search attained 0.75, and TPE reached 0.67. Policy-Based Reinforcement Learning exhibited performance changes after 39 trials, while Regularised Evolution optimised within 20 trials. The research investigates the optimisation dynamics and trade-offs between accuracy and model size for each strategy. Findings indicate that, in some cases, random search performed comparably to more complex algorithms when assessed using the M-factor. These results highlight how the M-factor addresses the limitations of existing metrics by guiding NAS towards balanced architectures, offering valuable insights for selecting strategies in scenarios requiring both performance and efficiency.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。