通过拓扑分析神经网络激活模式，揭示决策边界与损失变化规律
Topological Signatures of ReLU Neural Network Activation Patterns
- 用图的费德勒分割研究神经网络的分段结构
- 发现训练中多面体单元数量与损失下降趋势一致
- 适合对神经网络几何结构感兴趣的学者
本文研究了具有ReLU激活函数的前馈神经网络的激活模式所呈现的拓扑特征。重点分析由网络诱导的特征空间的多面体分解，考察其对偶图的费德勒分割，发现该分割与二分类任务中的决策边界存在相关性。此外，在回归任务中计算了细胞分解的同调结构，观察到训练过程中训练损失与多面体细胞数量的变化呈现出相似的动态行为。
原文摘要 · Abstract (English)
This paper explores the topological signatures of ReLU neural network activation patterns. We consider feedforward neural networks with ReLU activation functions and analyze the polytope decomposition of the feature space induced by the network. Mainly, we investigate how the Fiedler partition of the dual graph and show that it appears to correlate with the decision boundary -- in the case of binary classification. Additionally, we compute the homology of the cellular decomposition -- in a regression task -- to draw similar patterns in behavior between the training loss and polyhedral cell-count, as the model is trained.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。