用李普希茨约束让深度迭代模型更稳定，能保证收敛且外推到更难问题。
Rethinking Deep Thinking: Stable Learning of Algorithms using Lipschitz Constraints
- 引入李普希茨约束控制中间表示增长，提升训练稳定性
- 模型参数更少，推理时可保证唯一解收敛，成功率超传统方法
- 适用于难以学习的NP难题，如旅行商问题，具备强外推能力
迭代算法通过逐步计算直至求解。深度思维（Deep Thinking, DT）网络通过循环计算和卷积实现可扩展的迭代算法学习，但训练时常不稳定，且无法保证求解时的收敛与终止。本文通过分析中间表示的增长规律，构建了带李普希茨约束的深度思维模型（DT-L），参数更少，求解更可靠。此外，该框架在推理时保证学习到的迭代过程收敛至唯一解。我们在旅行商问题（TSP）上进行了基准测试，验证了改进系统在原版DT失败的NP难问题上仍能有效学习并外推至比训练集更复杂的问题。
原文摘要 · Abstract (English)
Iterative algorithms solve problems by taking steps until a solution is reached. Models in the form of Deep Thinking (DT) networks have been demonstrated to learn iterative algorithms in a way that can scale to different sized problems at inference time using recurrent computation and convolutions. However, they are often unstable during training, and have no guarantees of convergence/termination at the solution. This paper addresses the problem of instability by analyzing the growth in intermediate representations, allowing us to build models (referred to as Deep Thinking with Lipschitz Constraints (DT-L)) with many fewer parameters and providing more reliable solutions. Additionally our DT-L formulation provides guarantees of convergence of the learned iterative procedure to a unique solution at inference time. We demonstrate DT-L is capable of robustly learning algorithms which extrapolate to harder problems than in the training set. We benchmark on the traveling salesperson problem to evaluate the capabilities of the modified system in an NP-hard problem where DT fails to learn.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。