用隐函数定理解释神经网络权重如何编码数据语义。
Ensuring Semantics in Weights of Implicit Neural Representations through the Implicit Function Theorem
- 基于隐函数定理建立数据空间与权重表示空间的严格映射关系。
- 通过共享超网络将实例嵌入映射为INR权重，在2D/3D任务上表现媲美基线。
- 为权重学习提供理论支撑，适合研究元学习与隐式表示的学者。
权重空间学习（WSL）将神经网络权重视为一种数据模态，是元学习和迁移学习等任务的新兴方向。隐式神经表示（INRs）为此提供了理想实验平台，其中每组权重定义了一个从坐标到上下文值的映射，对应一个特定数据样本。然而，现有研究仍缺乏对数据语义如何编码进网络权重的精确理论解释。本文利用隐函数定理（IFT），建立了数据空间与其潜在权重表示空间之间的严格映射关系。我们分析了一种通过共享超网络将特定实例嵌入映射为INR权重的框架，在二维和三维数据集上的下游分类任务中表现达到现有基线水平。这些发现为未来关于网络权重的研究提供了理论视角。
原文摘要 · Abstract (English)
Weight Space Learning (WSL), which frames neural network weights as a data modality, is an emerging field with potential for tasks like meta-learning or transfer learning. Particularly, Implicit Neural Representations (INRs) provide a convenient testbed, where each set of weights determines the corresponding individual data sample as a mapping from coordinates to contextual values. So far, a precise theoretical explanation for the mechanism of encoding semantics of data into network weights is still missing. In this work, we deploy the Implicit Function Theorem (IFT) to establish a rigorous mapping between the data space and its latent weight representation space. We analyze a framework that maps instance-specific embeddings to INR weights via a shared hypernetwork, achieving performance competitive with existing baselines on downstream classification tasks across 2D and 3D datasets. These findings offer a theoretical lens for future investigations into network weights.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。