MoENAS自动设计更准确、公平、鲁棒的边缘神经网络。
MoENAS: Mixture-of-Expert based Neural Architecture Search for jointly Accurate, Fair, and Robust Edge Deep Neural Networks
- 基于专家混合架构搜索，自动优化模型性能
- 皮肤色调误差从14.09%降至5.60%，准确率提升4.02%
- 适合关注公平性与鲁棒性的边缘AI研发人员
近年来，针对边缘深度神经网络（DNNs）的优化主要集中在精度与效率上，传统方法如剪枝及近年的自动化设计均未充分考虑公平性、鲁棒性和泛化能力。在使用FACET数据集评估当前最优边缘DNNs时，发现其在10种不同肤色间的准确率差异高达14.09%，同时存在非鲁棒性与泛化能力差的问题。为此，我们提出基于专家混合的神经架构搜索（MoENAS），在专家混合空间中自动搜索兼具高精度、高公平性、强鲁棒性与良好泛化能力的边缘DNN。相比现有最优模型，MoENAS提升准确率4.02%，将肤色准确率差异从14.09%降至5.60%，鲁棒性提升3.80%，过拟合控制在0.21%，模型规模仅增加0.4M，接近当前主流水平。该方法为边缘DNN设计树立新基准，推动更包容、更可靠的边缘智能发展。
原文摘要 · Abstract (English)
There has been a surge in optimizing edge Deep Neural Networks (DNNs) for accuracy and efficiency using traditional optimization techniques such as pruning, and more recently, employing automatic design methodologies. However, the focus of these design techniques has often overlooked critical metrics such as fairness, robustness, and generalization. As a result, when evaluating SOTA edge DNNs' performance in image classification using the FACET dataset, we found that they exhibit significant accuracy disparities (14.09%) across 10 different skin tones, alongside issues of non-robustness and poor generalizability. In response to these observations, we introduce Mixture-of-Experts-based Neural Architecture Search (MoENAS), an automatic design technique that navigates through a space of mixture of experts to discover accurate, fair, robust, and general edge DNNs. MoENAS improves the accuracy by 4.02% compared to SOTA edge DNNs and reduces the skin tone accuracy disparities from 14.09% to 5.60%, while enhancing robustness by 3.80% and minimizing overfitting to 0.21%, all while keeping model size close to state-of-the-art models average size (+0.4M). With these improvements, MoENAS establishes a new benchmark for edge DNN design, paving the way for the development of more inclusive and robust edge DNNs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。