用模型预测控制统一反向传播与前向-前向算法，实现训练效率与性能的可调控平衡。
Unifying back-propagation and forward-forward algorithms through model predictive control
- 以模型预测控制框架统一反向传播与前向-前向算法
- 在深度线性网络上精确分析了预测范围对性能与效率的权衡
- 提出根据目标和模型特性选择最优优化步长的方法，适用于多种任务
我们提出一种基于模型预测控制（MPC）的深度神经网络训练框架，系统性地统一了反向传播（BP）与前向-前向（FF）算法。该框架生成一系列具有不同前瞻视野的中间训练算法，形成性能与效率之间的权衡。我们在深度线性网络上对这一权衡进行了精确分析，其定性结论可推广至一般网络。基于此分析，我们提出了一个基于给定目标与模型规格的优化视野选择原则。在多种模型与任务上的数值结果验证了该方法的通用性。
原文摘要 · Abstract (English)
We introduce a Model Predictive Control (MPC) framework for training deep neural networks, systematically unifying the Back-Propagation (BP) and Forward-Forward (FF) algorithms. At the same time, it gives rise to a range of intermediate training algorithms with varying look-forward horizons, leading to a performance-efficiency trade-off. We perform a precise analysis of this trade-off on a deep linear network, where the qualitative conclusions carry over to general networks. Based on our analysis, we propose a principled method to choose the optimization horizon based on given objectives and model specifications. Numerical results on various models and tasks demonstrate the versatility of our method.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。