让Transformer根据输入自动调节计算量,节省资源还保持精度。
Transformer^-1: Input-Adaptive Computation for Resource-Constrained Deployment
- 用双层控制机制动态决定每条数据的计算路径。
- 图像识别任务中减少42.7%算力、34.1%内存占用,精度几乎不变。
- 适合嵌入式设备部署,已在Jetson平台验证有效。
针对深度学习模型在动态场景下因固定计算模式导致的资源浪费问题,本文提出基于深层自适应原理的Transformer⁻¹架构。该架构通过建立复杂度与计算量的联合优化模型,实现输入特征与计算资源的动态匹配。核心贡献包括:(1) 设计包含复杂度预测器和强化学习策略网络的两层控制机制,支持端到端计算路径优化;(2) 推导出动态计算的下界理论,证明系统可逼近最优效率;(3) 提出层折叠技术与CUDA Graph预编译方案,解决动态架构的工程瓶颈。ImageNet-1K基准测试显示,相比标准Transformer,本方法降低42.7% FLOPs与34.1%峰值内存使用量,精度仅下降±0.3%。在Jetson AGX Xavier平台的实际部署验证了其在资源受限环境中的有效性。进一步在多个自然语言处理任务上测试,均显著提升资源效率。
原文摘要 · Abstract (English)
Addressing the resource waste caused by fixed computation paradigms in deep learning models under dynamic scenarios, this paper proposes a Transformer$^{-1}$ architecture based on the principle of deep adaptivity. This architecture achieves dynamic matching between input features and computational resources by establishing a joint optimization model for complexity and computation. Our core contributions include: (1) designing a two-layer control mechanism, composed of a complexity predictor and a reinforcement learning policy network, enabling end-to-end optimization of computation paths; (2) deriving a lower bound theory for dynamic computation, proving the system's theoretical reach to optimal efficiency; and (3) proposing a layer folding technique and a CUDA Graph pre-compilation scheme, overcoming the engineering bottlenecks of dynamic architectures. In the ImageNet-1K benchmark test, our method reduces FLOPs by 42.7\% and peak memory usage by 34.1\% compared to the standard Transformer, while maintaining comparable accuracy ($\pm$0.3\%). Furthermore, we conducted practical deployment on the Jetson AGX Xavier platform, verifying the effectiveness and practical value of this method in resource-constrained environments. To further validate the generality of the method, we also conducted experiments on several natural language processing tasks and achieved significant improvements in resource efficiency.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。