通过投影正则化实现近似等变性，提升模型效率与泛化能力。
Approximate Equivariance via Projection-based Regularisation
- 基于线性层的等变与非等变分量投影，全局惩罚非等变性。
- 在SO(3)等连续群上，性能优于采样正则化方法，且推理更快。
- 适用于需要物理一致性又容忍微小对称破缺的场景。
等变性是神经网络中强大的归纳偏置，可提升泛化性和物理一致性。然而，由于实际应用中对称性不完美以及非等变模型具备更优运行效率，近似等变模型逐渐受到关注。现有方法多依赖训练时的数据增强进行样本级正则化，尤其在连续群如SO(3)上样本复杂度高。本文提出一种基于投影的正则化方法，利用线性层在等变与非等变分量上的正交分解，在整个群轨道上以算子层面惩罚非等变性，而非逐点处理。我们构建了数学框架，可高效精确地在空域和谱域计算非等变惩罚项。实验表明，该方法在模型性能与效率上均优于现有近似等变方法，相较采样正则化实现显著的运行时间提升。
原文摘要 · Abstract (English)
Equivariance is a powerful inductive bias in neural networks, improving generalisation and physical consistency. Recently, however, non-equivariant models have regained attention, due to their better runtime performance and imperfect symmetries that might arise in real-world applications. This has motivated the development of approximately equivariant models that strike a middle ground between respecting symmetries and fitting the data distribution. Existing approaches in this field usually apply sample-based regularisers which depend on data augmentation at training time, incurring a high sample complexity, in particular for continuous groups such as $SO(3)$. This work instead approaches approximate equivariance via a projection-based regulariser which leverages the orthogonal decomposition of linear layers into equivariant and non-equivariant components. In contrast to existing methods, this penalises non-equivariance at an operator level across the full group orbit, rather than point-wise. We present a mathematical framework for computing the non-equivariance penalty exactly and efficiently in both the spatial and spectral domain. In our experiments, our method consistently outperforms prior approximate equivariance approaches in both model performance and efficiency, achieving substantial runtime gains over sample-based regularisers.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。