提出无需解析解的浅化层合并策略，高效压缩模型且不增加卷积核大小。
From Deep to Shallow: Unconstrained and Efficient Layer Merging Strategy
- 通过新策略实现无解析解时的层合并
- 在多个架构上验证，嵌入式平台推理速度提升显著
- 适合资源受限设备部署，代码已开源
尽管深度神经网络已在机器学习诸多领域成为基础，但高计算需求限制了其在资源受限环境中的应用。为解决此问题，已有深度压缩方法通过识别并线性化冗余激活函数，实现无中间非线性的层合并。然而，这些方法面临两大挑战：无法直接应用于带填充的卷积层（因缺乏解析解），且通常会增大合并后层的卷积核尺寸，从而限制加速效果。为此，本文提出一种高效策略，可在无解析解情况下实现层合并，且不增加卷积核大小。我们在多个模型架构和数据集上验证该方法，并在真实嵌入式平台上测量推理加速效果。代码已公开于 https://github.com/ShulzhenkoPetr/deep-to-shallow。
原文摘要 · Abstract (English)
Although Deep Neural Networks have become foundational in many areas of Machine Learning, high computational demands limit their application in resource-constrained environments. To address this issue, depth compression methods have been proposed to identify and linearize redundant activation functions, thereby allowing for the merging of layers without intermediate non-linearities. However, these methods face two key challenges: they cannot be directly applied to convolutions with padding due to the absence of an analytical solution for merging these layers, and they typically increase the kernel size of merged layers, thus limiting speed-up gains. To overcome these limitations, we propose an efficient strategy that enables merging of layers without an existing analytical solution, and also without increasing kernel size. We validate our approach across multiple architectures and datasets, and measure inference speed-up gains on real embedded platforms. We publicly released the code at https://github.com/ShulzhenkoPetr/deep-to-shallow.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。