arXiv:2604.09716cs.CVcs.AI2026-04

用动力系统视角分析模型训练,揭示隐藏的内部变化模式。

Training Deep Visual Networks Beyond Loss and Accuracy Through a Dynamical Systems Approach

论文配图:Training Deep Visual Networks Beyond Loss and Accuracy Through a Dynamical Systems Approach
图 1 · 摘自论文原文
  • 从层激活中提取三种动力学指标:整合度、亚稳态和稳定性指数。
  • CIFAR-10比CIFAR-100的整合度更高,反映任务难度差异。
  • 稳定性波动可提前预示收敛,适合研究训练机制的学者。

深度视觉识别模型通常仅通过损失和准确率来训练与评估,这些指标虽能反映模型是否改进,却难以揭示其内部表征在训练过程中的变化。本文提出一种新方法,借鉴信号分析技术(原用于生物神经活动研究),从训练各轮次的层激活中定义三个指标:反映跨层长程协调的整合度、捕捉网络在同步与非同步状态间灵活切换能力的亚稳态度,以及综合动力学稳定性指数。该框架应用于九种模型架构与数据集组合,包括ResNet变体、DenseNet-121、MobileNetV2、VGG-16及预训练Vision Transformer在CIFAR-10和CIFAR-100上的表现。结果表明:第一,整合度能稳定区分较简单的CIFAR-10与较难的CIFAR-100;第二,稳定性指数的波动变化可能在准确率完全平缓前就提示收敛;第三,整合度与亚稳态的关系映射出不同的训练行为风格。该研究为超越损失与准确率理解深度训练提供了探索性但有前景的新视角。

原文摘要 · Abstract (English)

Deep visual recognition models are usually trained and evaluated using metrics such as loss and accuracy. While these measures show whether a model is improving, they reveal very little about how its internal representations change during training. This paper introduces a complementary way to study that process by examining training through the lens of dynamical systems. Drawing on ideas from signal analysis originally used to study biological neural activity, we define three measures from layer activations collected across training epochs: an integration score that reflects long-range coordination across layers, a metastability score that captures how flexibly the network shifts between more and less synchronised states, and a combined dynamical stability index. We apply this framework to nine combinations of model architecture and dataset, including several ResNet variants, DenseNet-121, MobileNetV2, VGG-16, and a pretrained Vision Transformer on CIFAR-10 and CIFAR-100. The results suggest three main patterns. First, the integration measure consistently distinguishes the easier CIFAR-10 setting from the more difficult CIFAR-100 setting. Second, changes in the volatility of the stability index may provide an early sign of convergence before accuracy fully plateaus. Third, the relationship between integration and metastability appears to reflect different styles of training behaviour. Overall, this study offers an exploratory but promising new way to understand deep visual training beyond loss and accuracy.

动力系统训练分析模型表征

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。