用拓扑分析方法揭示神经网络损失曲面的深层结构，发现传统方法忽略的平滑性变化。
Landscaper: Understanding Loss Landscapes Through Multi-Dimensional Topological Analysis
- 结合海森子空间与拓扑数据分析，捕捉损失曲面的层级与连通性特征
- 提出SMAD指标量化曲面平滑度，识别训练过程中的简化现象
- 适用于语言模型和小样本科学建模，助力模型诊断与架构优化
损失曲面是理解神经网络优化与泛化的重要工具，但传统低维分析常遗漏复杂的拓扑结构。本文提出Landscaper，一个用于任意维度损失曲面分析的开源Python工具包。该工具结合基于海森矩阵的子空间构建与拓扑数据分析，揭示如基底层级、连通性等几何结构。核心贡献为鞍点-极小值平均距离（SMAD），用于量化曲面平滑度。实验表明，在多种架构与任务中，包括预训练语言模型，SMAD能捕捉训练过程中的景观简化等转变，而传统指标无法察觉。此外，在化学性质预测等数据稀缺的科学机器学习任务中，SMAD可作为分布外泛化能力的度量，为模型诊断与架构设计提供关键洞察。
原文摘要 · Abstract (English)
Loss landscapes are a powerful tool for understanding neural network optimization and generalization, yet traditional low-dimensional analyses often miss complex topological features. We present Landscaper, an open-source Python package for arbitrary-dimensional loss landscape analysis. Landscaper combines Hessian-based subspace construction with topological data analysis to reveal geometric structures such as basin hierarchy and connectivity. A key component is the Saddle-Minimum Average Distance (SMAD) for quantifying landscape smoothness. We demonstrate Landscaper's effectiveness across various architectures and tasks, including those involving pre-trained language models, showing that SMAD captures training transitions, such as landscape simplification, that conventional metrics miss. We also illustrate Landscaper's performance in challenging chemical property prediction tasks, where SMAD can serve as a metric for out-of-distribution generalization, offering valuable insights for model diagnostics and architecture design in data-scarce scientific machine learning scenarios.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。