新方法让神经网络训练更快更稳，减少无效步骤30%以上。
A Non-Monotone Preconditioned Trust-Region Method for Neural Network Training
- 用非单调准则和并行子域协同优化，提升训练效率。
- 相比旧方法，计算时间降30%，被拒绝的步骤减少至1/3。
- 适合大规模分布式训练，尤其对资源受限场景友好。
大规模神经网络训练可借助域分解技术，将网络拆分为多个子域并行训练，并通过全局信任域机制耦合。基于加性预条件信任域策略（APTS），本文提出一种非单调变体NAPTS，采用非线性加性Schwarz预条件器，融合并行子域修正与全局粗网格方向。引入窗口化接受准则，允许可控的目标函数上升，避免对有效粗步长的无谓拒绝。实验表明，NAPTS在保持精度的同时，使CPU时间减少30%，被拒绝的迭代步骤降至APTS的三分之一。
原文摘要 · Abstract (English)
Training deep neural networks at scale can benefit from domain decomposition, where the network is split into subdomains trained in parallel and coupled by a global trust-region mechanism. Building on the Additively Preconditioned Trust-Region Strategy (APTS), we propose a non-monotone variant with a nonlinear additive Schwarz preconditioner that combines parallel subdomain corrections with global coarse-space directions. A windowed acceptance criterion allows controlled objective increases, avoiding needless rejection of effective coarse steps. The resulting non-monotone APTS (NAPTS) preserves accuracy while reducing CPU time by 30\% and cutting rejected steps to one third of those in APTS.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。