发现科学机器学习的多种训练模式，揭示不同模式下的失效机制
Unveiling Multi-regime Patterns in SciML: Distinct Failure Modes and Regime-specific Optimization
- 构建诊断框架，联合分析性能、训练动态与损失曲面几何
- 识别出跨模型普遍存在的三种训练模式，优化效果因模式而异
- 揭示精细失效模式，为提升模型鲁棒性提供针对性指导
神经网络在不同超参数设置下会进入不同的训练'模式'，同一模式内行为一致，不同模式间有显著差异。本文通过一种面向模式的诊断框架，联合分析性能、训练动态与损失曲面几何，研究科学机器学习（SciML）模型中的多模式行为。发现三个关键结果：(i) 多种标准SciML模型、约束方式与优化器设计下均出现稳定的三模式结构；(ii) 优化有效性具有模式特异性，无单一方法在所有模式中表现优异；(iii) SciML模型可表现出精细的失效模式，挑战传统损失曲面度量的解释。研究结果为建立任务无关的失效模式统一视角，以及实现模式感知的优化指导提供了方法支持。在包含物理信息神经网络、神经算子和神经常微分方程等广泛使用的SciML模型上，基于代表性常微分方程与偏微分方程基准进行了验证。
原文摘要 · Abstract (English)
Neural networks trained under different hyperparameter settings can fall into distinct training "regimes," with consistent behavior within regimes and qualitative differences across regimes. In this paper, we study such multi-regime behavior in scientific machine learning (SciML) models through a regime-aware diagnostic framework that jointly analyzes performance, training dynamics, and loss-landscape geometry. We identify three key findings: (i) a consistent three-regime structure emerges across many standard SciML models, different constraint enforcements, and various optimizer designs; (ii) optimization effectiveness is regime-specific, with no single method performing well across all regimes; and (iii) SciML models can exhibit fine-grained failure modes that can challenge conventional interpretations of standard loss-landscape metrics. Our results provide an approach to establish a unified, task-oblivious perspective on failure modes in SciML and to inform regime-aware guidance for improving robustness. We validate these findings across widely-used SciML models, including physics-informed neural networks, neural operators, and neural ordinary differential equations, on benchmarks spanning representative ordinary and partial differential equations.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。