发现深度神经网络的结构混乱度低于随机模型，暗示其功能更有序。
Computing frustration and near-monotonicity in deep neural networks
- 用符号图计算网络的混乱程度（frustration）
- 所有预训练网络的混乱度均低于随机模型预期
- 混乱度低意味着网络函数近似单调，适合研究模型隐式正则化
针对深度神经网络的符号图，可计算其混乱度（frustration），即衡量图结构与完全平衡状态的偏离程度。在所考察的所有预训练深度卷积神经网络中，其混乱度始终低于随机模型的预期水平。从统计物理角度看，这表明网络所编码的无序程度低于随机模型，类似于伊辛自旋玻璃模型中的有序态。从功能角度而言，低混乱度意味着网络函数表现出近单调性，即其行为更接近单调函数，而非随机模型中的复杂波动。我们观察到所有网络在由混乱度决定的偏序关系下均呈现近单调趋势。这表明深度卷积神经网络的整体行为比随机模型更有序，提示了一种新的隐式正则化机制。
原文摘要 · Abstract (English)
For the signed graph associated to a deep neural network, one can compute the frustration level, i.e., test how close or distant the graph is to structural balance. For all the pretrained deep convolutional neural networks we consider, we find that the frustration is always less than expected from null models. From a statistical physics point of view, and in particular in reference to an Ising spin glass model, the reduced frustration indicates that the amount of disorder encoded in the network is less than in the null models. From a functional point of view, low frustration (i.e., proximity to structural balance) means that the function representing the network behaves near-monotonically, i.e., more similarly to a monotone function than in the null models. Evidence of near-monotonic behavior along the partial order determined by frustration is observed for all networks we consider. This confirms that the class of deep convolutional neural networks tends to have a more ordered behavior than expected from null models, and suggests a novel form of implicit regularization.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。