让普通神经网络学会近似对称性,速度更快效果不差。
Relaxed Equivariance via Multitask Learning
- 用多任务学习在普通网络上训练近似对称性
- 相比严格对称模型快10倍,性能相当
- 适合需要快速推理且对称性非严格要求的场景
将等变性作为归纳偏置引入深度学习架构,可有效利用数据对称性,在化学和动力系统等领域表现良好。特别是旋转平移对称性对建模几何图和分子3D结构至关重要,有助于提升泛化能力。然而,严格等变模型因计算复杂度高常难应用。本文提出REMUL训练方法,通过多任务学习在无约束网络上学习近似等变性。将等变性作为可调节目标与主任务损失联合优化,实现对近似对称程度的定量控制,放松了传统等变架构的刚性约束。实验表明,无约束模型通过最小化额外的简单等变损失,可有效学习近似对称性,实现等变约束与任务性能之间的灵活权衡。该方法在性能上媲美等变基线,推理速度提升达10倍,训练速度提升2.5倍,为在无约束架构中实用化对称性提供了高效可行方案。
原文摘要 · Abstract (English)
Incorporating equivariance as an inductive bias into deep learning architectures to take advantage of the data symmetry has been successful in multiple applications, such as chemistry and dynamical systems. In particular, roto-translations are crucial for effectively modeling geometric graphs and molecules, where understanding the 3D structures enhances generalization. However, strictly equivariant models often pose challenges due to their higher computational complexity. In this paper, we introduce REMUL, a training procedure that learns \emph{approximate} equivariance for unconstrained networks via multitask learning. By formulating equivariance as a tunable objective alongside the primary task loss, REMUL offers a principled way to control the degree of approximate symmetry, relaxing the rigid constraints of traditional equivariant architectures. We show that unconstrained models (which do not build equivariance into the architecture) can learn approximate symmetries by minimizing an additional simple equivariance loss. This enables quantitative control over the trade-off between enforcing equivariance constraints and optimizing for task-specific performance. Our method achieves competitive performance compared to equivariant baselines while being significantly faster (up to 10$\times$ at inference and 2.5$\times$ at training), offering a practical and adaptable approach to leveraging symmetry in unconstrained architectures.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。