通过最大化内存利用提升模型泛化能力，实现更高效的神经网络部署。
Memory-Optimized Once-For-All Network
- 构建记忆优化的Once-For-All超网络，动态分配各层内存使用
- ImageNet上相较原OFA提升内存利用率与模型准确率
- 适合资源受限设备上的高效模型部署
在不同硬件平台部署深度神经网络（DNN）面临资源约束差异的挑战。尽管手工设计方法致力于使模型适配硬件，神经架构搜索（NAS）正成为无需牺牲性能即可构建更高效DNN的工具。其中，一次性全貌（Once-For-All, OFA）方法通过从单一超网络中采样高性能子网络，显著降低计算开销。然而，OFA仅限制每层最大内存使用，未充分挖掘目标设备的内存潜力，导致模型泛化能力未被完全释放。本文提出一种内存优化的OFA（MOOFA）超网络，旨在通过最大化不同配置下的内存使用（如特征多样性）来提升资源受限设备上的模型部署效果。在ImageNet上的实验表明，相比原始OFA超网络，MOOFA在内存利用率和模型准确率方面均有提升。代码已开源：https://github.com/MaximeGirard/memory-optimized-once-for-all。
原文摘要 · Abstract (English)
Deploying Deep Neural Networks (DNNs) on different hardware platforms is challenging due to varying resource constraints. Besides handcrafted approaches aiming at making deep models hardware-friendly, Neural Architectures Search is rising as a toolbox to craft more efficient DNNs without sacrificing performance. Among these, the Once-For-All (OFA) approach offers a solution by allowing the sampling of well-performing sub-networks from a single supernet -- this leads to evident advantages in terms of computation. However, OFA does not fully utilize the potential memory capacity of the target device, focusing instead on limiting maximum memory usage per layer. This leaves room for an unexploited potential in terms of model generalizability. In this paper, we introduce a Memory-Optimized OFA (MOOFA) supernet, designed to enhance DNN deployment on resource-limited devices by maximizing memory usage (and for instance, features diversity) across different configurations. Tested on ImageNet, our MOOFA supernet demonstrates improvements in memory exploitation and model accuracy compared to the original OFA supernet. Our code is available at https://github.com/MaximeGirard/memory-optimized-once-for-all.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。