arXiv:2409.20447cs.LGcs.AI2024-09被引 2

POMONAG同时优化模型精度与效率,生成更优的神经网络架构。

POMONAG: Pareto-Optimal Many-Objective Neural Architecture Generator

  • 基于扩散模型的多目标优化,兼顾精度、参数量、计算量和推理延迟。
  • 在15个图像分类数据集上超越现有方法,生成帕累托最优架构。
  • 适合需要高效部署的边缘计算场景,对资源受限环境友好。

神经架构搜索(NAS)自动化设计神经网络,减少对人工经验的依赖。尽管当前NAS方法计算成本高且依赖特定数据集,辅助预测器可减少需训练的模型数量,从而降低搜索时间。该策略被用于生成满足多重计算约束的架构。近期出现的可迁移NAS将搜索过程从数据集依赖转向任务依赖。其中,DiffusionNAG是先进方法,其基于扩散模型的方法能高效生成在未见数据集上表现优异的架构,无需进一步适配。然而,该方法仅关注精度,忽略了模型复杂度、计算效率和推理延迟等关键因素,而这些在资源受限环境中至关重要。本文提出帕累托最优多目标神经架构生成器(POMONAG),通过多目标扩散过程扩展DiffusionNAG。POMONAG同时优化精度、参数量、乘加操作(MACs)和推理延迟,并集成性能预测器估算各项指标以引导扩散梯度。通过扩充训练元数据集、应用帕累托前沿过滤及优化条件嵌入,显著提升生成效果。在NASBench201与MobileNetV3两个搜索空间上,针对15个图像分类数据集的验证表明,POMONAG生成的架构在性能与效率上均优于现有最先进方法。

原文摘要 · Abstract (English)

Neural Architecture Search (NAS) automates neural network design, reducing dependence on human expertise. While NAS methods are computationally intensive and dataset-specific, auxiliary predictors reduce the models needing training, decreasing search time. This strategy is used to generate architectures satisfying multiple computational constraints. Recently, Transferable NAS has emerged, generalizing the search process from dataset-dependent to task-dependent. In this field, DiffusionNAG is a state-of-the-art method. This diffusion-based approach streamlines computation, generating architectures optimized for accuracy on unseen datasets without further adaptation. However, by focusing solely on accuracy, DiffusionNAG overlooks other crucial objectives like model complexity, computational efficiency, and inference latency -- factors essential for deploying models in resource-constrained environments. This paper introduces the Pareto-Optimal Many-Objective Neural Architecture Generator (POMONAG), extending DiffusionNAG via a many-objective diffusion process. POMONAG simultaneously considers accuracy, number of parameters, multiply-accumulate operations (MACs), and inference latency. It integrates Performance Predictor models to estimate these metrics and guide diffusion gradients. POMONAG's optimization is enhanced by expanding its training Meta-Dataset, applying Pareto Front Filtering, and refining embeddings for conditional generation. These enhancements enable POMONAG to generate Pareto-optimal architectures that outperform the previous state-of-the-art in performance and efficiency. Results were validated on two search spaces -- NASBench201 and MobileNetV3 -- and evaluated across 15 image classification datasets.

神经架构搜索多目标优化扩散模型高效部署

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。