解析图元网络的内部表示，发现其与普通网络有本质差异
On the Internal Representations of Graph Metanetworks
- 用中心核对齐分析元网络参数空间的表征特性
- 图元网络的表示空间与普通MLP/CNN显著不同
- 为理解权重空间学习提供新视角，适合研究元学习者
权重空间学习是深度学习领域的新兴范式，其核心目标是利用专门设计的神经网络（即元网络）从一组参数中提取信息特征。然而，元网络仅从参数中学习的机制仍不明确。为此，本文首次系统研究图元网络（GMNs）的内部表示，采用中心核对齐（CKA）方法进行分析。通过一系列实验，揭示了图元网络与通用神经网络（如多层感知机MLPs和卷积神经网络CNNs）在表示空间上的本质差异，为理解权重空间学习提供了新的理论依据。
原文摘要 · Abstract (English)
Weight space learning is an emerging paradigm in the deep learning community. The primary goal of weight space learning is to extract informative features from a set of parameters using specially designed neural networks, often referred to as \emph{metanetworks}. However, it remains unclear how these metanetworks learn solely from parameters. To address this, we take the first step toward understanding \emph{representations} of metanetworks, specifically graph metanetworks (GMNs), which achieve state-of-the-art results in this field, using centered kernel alignment (CKA). Through various experiments, we reveal that GMNs and general neural networks (\textit{e.g.,} multi-layer perceptrons (MLPs) and convolutional neural networks (CNNs)) differ in terms of their representation space.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。