提出新方法让对称网络能打破对称性,提升生成与预测性能。
Improving Equivariant Networks with Probabilistic Symmetry Breaking
- 用随机化规范化的思路,在保持对称性先验的前提下打破对称性
- 在图扩散、自编码器和自旋系统上显著提升模型性能
- 适合需要对称性又需破对称的生成模型与图神经网络
等变性将已知对称性嵌入神经网络,常可提升泛化能力。然而等变网络无法打破对称性:其输出至少保留输入的自对称性。这在自对称性普遍的任务和生成模型中构成关键问题,后者必须从高度对称的潜在空间中重建。该根本限制可通过考虑等变条件分布而非等变函数来解决。本文给出表示此类分布的必要充分条件,并提出可实践的框架——通过随机化规范化在任意等变网络中实现对称性打破。所提方法SymPE(对称性打破位置编码)可解释为位置编码的扩展。该方法在保持对称性归纳偏置的同时扩大了表示能力,理论依据来自泛化界。实验表明,SymPE显著提升了群等变网络与图神经网络在图扩散模型、图自编码器及晶格自旋系统建模中的表现。
原文摘要 · Abstract (English)
Equivariance encodes known symmetries into neural networks, often enhancing generalization. However, equivariant networks cannot break symmetries: the output of an equivariant network must, by definition, have at least the same self-symmetries as the input. This poses an important problem, both (1) for prediction tasks on domains where self-symmetries are common, and (2) for generative models, which must break symmetries in order to reconstruct from highly symmetric latent spaces. This fundamental limitation can be addressed by considering equivariant conditional distributions, instead of equivariant functions. We present novel theoretical results that establish necessary and sufficient conditions for representing such distributions. Concretely, this representation provides a practical framework for breaking symmetries in any equivariant network via randomized canonicalization. Our method, SymPE (Symmetry-breaking Positional Encodings), admits a simple interpretation in terms of positional encodings. This approach expands the representational power of equivariant networks while retaining the inductive bias of symmetry, which we justify through generalization bounds. Experimental results demonstrate that SymPE significantly improves performance of group-equivariant and graph neural networks across diffusion models for graphs, graph autoencoders, and lattice spin system modeling.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。