用随机深度训练的模型实现无需额外开销的自适应推理。
Leveraging Stochastic Depth Training for Adaptive Inference
- 基于随机深度训练的模型更耐受推理时跳层，实现动态调整。
- 在仅损失0.71%准确率下，功耗效率提升最高达2倍。
- 适合资源受限场景下的高效推理部署，无需复杂控制逻辑。
动态深度神经网络优化技术如层跳过可提升灵活性与效率，但存在决策门导致内存开销大、训练复杂（如不可导操作）、性能-质量权衡难以控制等问题。本文提出一种更简单有效的自适应推理方案：零开销、单模型、时间可预测。核心思想是利用随机深度训练（Stochastic Depth）的模型在推理时对任意层跳过具有更强鲁棒性。我们先从随机训练的模型中筛选出近似帕累托最优的跳层配置，用于运行时动态调整。相比原始ResNet，该方法在准确率下降仅0.71%的前提下，功耗效率最高提升2倍。
原文摘要 · Abstract (English)
Dynamic DNN optimization techniques such as layer-skipping offer increased adaptability and efficiency gains but can lead to i) a larger memory footprint as in decision gates, ii) increased training complexity (e.g., with non-differentiable operations), and iii) less control over performance-quality trade-offs due to its inherent input-dependent execution. To approach these issues, we propose a simpler yet effective alternative for adaptive inference with a zero-overhead, single-model, and time-predictable inference. Central to our approach is the observation that models trained with Stochastic Depth -- a method for faster training of residual networks -- become more resilient to arbitrary layer-skipping at inference time. We propose a method to first select near Pareto-optimal skipping configurations from a stochastically-trained model to adapt the inference at runtime later. Compared to original ResNets, our method shows improvements of up to 2X in power efficiency at accuracy drops as low as 0.71%.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。