让神经网络自动对齐任意李群对称性,提升物理方程求解的泛化能力。
Lie Algebra Canonicalization: Equivariant Neural Operators under arbitrary Lie Groups
- 仅利用对称性的微小生成元,无需完整群结构即可实现等变性。
- 在预训练模型上通过输入归一化实现等变,兼容现有架构。
- 适用于图像分类和偏微分方程求解,尤其适合非紧致对称群场景。
追求鲁棒且可泛化的机器学习模型推动了利用对称性构建等变神经网络的研究。在偏微分方程(PDE)求解中,已有工作表明李点对称性可通过数据与损失增强为物理信息神经网络(PINNs)提供有益归纳偏置。然而,直接在模型架构中强制实现等变性仍具挑战,因许多PDE具有非紧致对称群,通常仅研究其无穷小生成元,导致与现有等变架构不兼容。本文提出李代数规范化(LieLAC),仅依赖对称群的无穷小生成元作用,避免对完整群结构的依赖。我们解决了规范化的理论问题,并建立了其与连续非紧致群下框架平均的联系。在规范框架下,LieLAC可轻松集成至无约束预训练模型中,将输入转换为规范形式后输入原模型,有效根据允许对称性对齐输入以进行推理。该方法使用标准李群下降算法,在预训练模型中实现等变性。我们在不变图像分类与李点对称性等变神经PDE求解任务中验证了其有效性。
原文摘要 · Abstract (English)
The quest for robust and generalizable machine learning models has driven recent interest in exploiting symmetries through equivariant neural networks. In the context of PDE solvers, recent works have shown that Lie point symmetries can be a useful inductive bias for Physics-Informed Neural Networks (PINNs) through data and loss augmentation. Despite this, directly enforcing equivariance within the model architecture for these problems remains elusive. This is because many PDEs admit non-compact symmetry groups, oftentimes not studied beyond their infinitesimal generators, making them incompatible with most existing equivariant architectures. In this work, we propose Lie aLgebrA Canonicalization (LieLAC), a novel approach that exploits only the action of infinitesimal generators of the symmetry group, circumventing the need for knowledge of the full group structure. To achieve this, we address existing theoretical issues in the canonicalization literature, establishing connections with frame averaging in the case of continuous non-compact groups. Operating within the framework of canonicalization, LieLAC can easily be integrated with unconstrained pre-trained models, transforming inputs to a canonical form before feeding them into the existing model, effectively aligning the input for model inference according to allowed symmetries. LieLAC utilizes standard Lie group descent schemes, achieving equivariance in pre-trained models. Finally, we showcase LieLAC's efficacy on tasks of invariant image classification and Lie point symmetry equivariant neural PDE solvers using pre-trained models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。