提出新方法连接独立训练的模型，支持更多网络架构且路径更稳定。
Connecting Independently Trained Modes via Layer-Wise Connectivity
- 基于分层连接思想，构建跨模型的低损失路径。
- 在多种新型网络（如MobileNet、EfficientNet）上实现稳定连接。
- 适用于不同超参数训练的模型，适合研究模型泛化性的人参考。
实证研究表明，在独立训练的神经网络模型间可以构建连续的低损失路径，这种现象称为模式连通性，指参数空间中不同最优解之间存在这样的路径。然而，现有方法难以可靠连接独立训练的模型，且主要在有限架构（如基础CNN、VGG、ResNet）上验证，对新模型的效果尚不明确。本文提出一种新算法，可推广至更广泛的网络结构，包括MobileNet、ShuffleNet、EfficientNet、RegNet、Deep Layer Aggregation（DLA）和Compact Convolutional Transformers（CCT）。该方法不仅适用范围更广，且在不同模型对间生成更一致的连通路径，还可连接由不同训练超参数得到的模型，显著提升可靠性。
原文摘要 · Abstract (English)
Empirical studies have shown that continuous low-loss paths can be constructed between independently trained neural network models. This phenomenon, known as mode connectivity, refers to the existence of such paths between distinct modes-i.e., well-trained solutions in parameter space. However, existing empirical methods do not reliably connect independently trained modes and have been evaluated mainly on a narrow set of architectures (e.g., basic CNNs, VGG, and ResNet), leaving their effectiveness on newer models unclear. In this work, we propose a new empirical algorithm for connecting independently trained modes that generalizes beyond traditional architectures and supports a broader range of networks, including MobileNet, ShuffleNet, EfficientNet, RegNet, Deep Layer Aggregation (DLA), and Compact Convolutional Transformers (CCT). In addition to broader applicability, the proposed method yields more consistent connectivity paths across independently trained mode pairs and supports connecting modes obtained with different training hyperparameters.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。