用低维隐向量替代高维权重，大幅减少参数量并防止过拟合。
Mapping Networks
- 用可训练的低维隐向量映射到网络权重空间，基于权重存在于低维流形的假设。
- 在图像分类和深度伪造检测等任务中，仅需原模型0.5%参数（约500倍压缩）即达相当性能。
- 适合追求高效训练、轻量化部署或研究模型压缩的开发者与研究人员。
现代深度学习模型参数量持续增长，带来了训练效率低下和过拟合问题。我们提出「映射网络」（Mapping Networks），基于大模型训练后的参数位于平滑低维流形上的假设，将高维权重空间替换为紧凑可训练的隐向量空间。通过专门设计的映射损失，理论与实践均证明该隐空间到目标权重空间存在有效映射。映射网络显著降低过拟合，在图像分类、深度伪造检测等复杂视觉与序列任务中表现媲美甚至优于原网络，同时将可训练参数减少至原模型的99.5%（约500倍压缩）。
原文摘要 · Abstract (English)
The escalating parameter counts in modern deep learning models pose a fundamental challenge to efficient training and resolution of overfitting. We address this by introducing the \emph{Mapping Networks} which replace the high dimensional weight space by a compact, trainable latent vector based on the hypothesis that the trained parameters of large networks reside on smooth, low-dimensional manifolds. Henceforth, the Mapping Theorem enforced by a dedicated Mapping Loss, shows the existence of a mapping from this latent space to the target weight space both theoretically and in practice. Mapping Networks significantly reduce overfitting and achieve comparable to better performance than target network across complex vision and sequence tasks, including Image Classification, Deepfake Detection etc, with $\mathbf{99.5\%}$, i.e., around $500\times$ reduction in trainable parameters.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。