提出评估前馈计算图好坏的两个关键指标，指导神经网络结构设计。
What makes a good feedforward computational graph?
- 引入保真度与混合时间衡量前馈图性能
- 发现两种指标互补，能预测模型训练效果
- 适用于想优化网络结构的研究者
正如大量关于图重布线的研究所示，神经网络所采用的计算图对下游性能有显著影响。某些与计算图相关的效应，如信息传递不足和过度压缩，甚至会使模型无法学习某些函数。这些效应此前主要在无向图领域被深入研究；然而近年来，前馈计算图——即无反向边的有向图——受到越来越多关注。本文研究前馈计算图的理想特性，发现两个重要且互补的度量：保真度与混合时间，并通过这两个度量评估了几种常见图结构。研究结合了各类图在渐近行为上的理论分析，以及将这些度量与使用对应图结构训练的神经网络模型性能之间的相关性验证。
原文摘要 · Abstract (English)
As implied by the plethora of literature on graph rewiring, the choice of computational graph employed by a neural network can make a significant impact on its downstream performance. Certain effects related to the computational graph, such as under-reaching and over-squashing, may even render the model incapable of learning certain functions. Most of these effects have only been thoroughly studied in the domain of undirected graphs; however, recent years have seen a significant rise in interest in feedforward computational graphs: directed graphs without any back edges. In this paper, we study the desirable properties of a feedforward computational graph, discovering two important complementary measures: fidelity and mixing time, and evaluating a few popular choices of graphs through the lens of these measures. Our study is backed by both theoretical analyses of the metrics' asymptotic behaviour for various graphs, as well as correlating these metrics to the performance of trained neural network models using the corresponding graphs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。