不依赖每样本真实权重,通过学习权重演化路径训练超网络。
HyperNet Fields: Efficiently Training Hypernetworks without Ground Truth by Learning Weight Trajectories
- 用收敛状态作为输入,让超网络建模权重整个训练轨迹
- 无需每样本真实权重,仅靠梯度一致性即可训练超网络
- 适用于个性化图像生成与3D形状重建,效果媲美有监督方法
为高效适配大模型或训练神经表示生成模型,超网络受到关注。然而,超网络训练繁琐,通常需每样本的真实优化权重。获取这些权重本身即是一个训练问题——需训练适配权重甚至整个神经场。本文提出一种无需任何样本级真实权重的超网络训练方法。核心思想是学习一个超网络场,并估计网络权重在整个训练过程中的演化轨迹,而非仅预测最终收敛状态。具体地,向超网络引入收敛状态作为额外输入,使其成为建模任务网络完整收敛路径的神经场。关键优势在于:在任意收敛状态下,估计权重的梯度必须与原始任务梯度一致,这一约束足以驱动超网络场的训练。我们在个性化图像生成和从图像、点云进行3D形状重建任务中验证了该方法的有效性,结果表明在无任何样本级真实权重的情况下仍能取得竞争性性能。
原文摘要 · Abstract (English)
To efficiently adapt large models or to train generative models of neural representations, Hypernetworks have drawn interest. While hypernetworks work well, training them is cumbersome, and often requires ground truth optimized weights for each sample. However, obtaining each of these weights is a training problem of its own-one needs to train, e.g., adaptation weights or even an entire neural field for hypernetworks to regress to. In this work, we propose a method to train hypernetworks, without the need for any per-sample ground truth. Our key idea is to learn a Hypernetwork `Field` and estimate the entire trajectory of network weight training instead of simply its converged state. In other words, we introduce an additional input to the Hypernetwork, the convergence state, which then makes it act as a neural field that models the entire convergence pathway of a task network. A critical benefit in doing so is that the gradient of the estimated weights at any convergence state must then match the gradients of the original task -- this constraint alone is sufficient to train the Hypernetwork Field. We demonstrate the effectiveness of our method through the task of personalized image generation and 3D shape reconstruction from images and point clouds, demonstrating competitive results without any per-sample ground truth.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。