用对称性增强的图元网络加速模型微调，一次完成。
Symmetry-Aware Fully-Amortized Optimization with Scale Equivariant Graph Metanetworks
- 设计尺度等变图元网络，直接在权重空间进行快速微调。
- 实验表明该方法比传统迭代优化更快，且在不同架构上表现更优。
- 揭示卷积网络因对称性更小而更易优化，适合高效神经网络调参者。
摊销优化通过学习共享结构来加速相关优化问题的求解。本文探索使用尺度等变图元网络（Scale Equivariant Graph Metanetworks, ScaleGMNs）实现这一目标。通过直接在权重空间操作，ScaleGMNs 支持单次完成现有模型的微调，减少对迭代优化的需求。我们通过实验证明了该方法的有效性，并提供了理论结果：与多层感知机相比，卷积神经网络中由缩放对称性引起的规范自由度严格更小。这一洞察有助于解释本文及 Kalogeropoulos 等人（2024）观察到的架构间性能差异。总体而言，我们的研究凸显了对称性感知元网络在高效、可泛化神经网络优化中的潜力。开源代码：https://github.com/daniuyter/scalegmn_amortization
原文摘要 · Abstract (English)
Amortized optimization accelerates the solution of related optimization problems by learning mappings that exploit shared structure across problem instances. We explore the use of Scale Equivariant Graph Metanetworks (ScaleGMNs) for this purpose. By operating directly in weight space, ScaleGMNs enable single-shot fine-tuning of existing models, reducing the need for iterative optimization. We demonstrate the effectiveness of this approach empirically and provide a theoretical result: the gauge freedom induced by scaling symmetries is strictly smaller in convolutional neural networks than in multi-layer perceptrons. This insight helps explain the performance differences observed between architectures in both our work and that of Kalogeropoulos et al. (2024). Overall, our findings underscore the potential of symmetry-aware metanetworks as a powerful approach for efficient and generalizable neural network optimization. Open-source code: https://github.com/daniuyter/scalegmn_amortization
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。