从损失曲面几何视角梳理模型融合，揭示其与可解释性的深层联系。
Deep Model Merging: The Sister of Neural Network Interpretability -- A Survey
- 基于损失曲面几何分析模型融合的四大特性：模式凸性、确定性、方向性与连通性。
- 发现模型融合中的重复现象可归因于神经网络训练的内在结构规律。
- 为模型可解释性与鲁棒性研究提供新方向，适合关注模型理解的研究者。
我们从损失曲面几何的角度综述模型融合研究，将模型融合的实证观察与损失曲面分析的结果关联到神经网络训练的本质规律及其内部表征的形成机制。我们将该领域文献中反复出现的实证现象归纳为损失曲面几何的四个核心特征：模式凸性、确定性、方向性和连通性。我们认为，模型融合所揭示的表征结构洞见对模型可解释性与鲁棒性具有应用价值，并在这些领域的交叉处提出若干有前景的新研究方向。
原文摘要 · Abstract (English)
We survey the model merging literature through the lens of loss landscape geometry to connect observations from empirical studies on model merging and loss landscape analysis to phenomena that govern neural network training and the emergence of their inner representations. We distill repeated empirical observations from the literature in these fields into descriptions of four major characteristics of loss landscape geometry: mode convexity, determinism, directedness, and connectivity. We argue that insights into the structure of learned representations from model merging have applications to model interpretability and robustness, subsequently we propose promising new research directions at the intersection of these fields.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。