arXiv:2410.21853cs.LGcs.AI2024-10NeurIPS被引 15

从数据中自动学习连续对称性,提升模型效率与泛化能力。

Learning Infinitesimal Generators of Continuous Symmetries from Data

  • 基于向量场生成的连续变换,无需预设对称群
  • 通过可微有效性评分寻找数据内在对称性
  • 适用于图像与偏微分方程,支持非线性对称

数据中的对称性可显著提升学习效率与模型泛化能力。当数据显式呈现对称结构时,可自然指导模型设计或学习策略。然而在多数真实场景中,数据分布中的具体对称性往往难以明确识别。现有方法虽尝试数据驱动学习对称性,但通常依赖预定义的李群,多限于线性或仿射变换。本文提出一种新算法,基于单参数群定义的连续变换,其沿向量场方向流动,由无穷小生成元刻画。该方法仅包含最小归纳偏置,不仅涵盖常见的李群对称性,还可扩展至非线性生成元导出的对称性。为学习这些对称性,引入可微且易计算的有效性评分,用于判断变换后数据是否仍符合任务要求。该评分支持高效搜索数据内在对称性。我们在图像数据和偏微分方程两个领域验证了方法优势。代码已开源。

原文摘要 · Abstract (English)

Exploiting symmetry inherent in data can significantly improve the sample efficiency of a learning procedure and the generalization of learned models. When data clearly reveals underlying symmetry, leveraging this symmetry can naturally inform the design of model architectures or learning strategies. Yet, in numerous real-world scenarios, identifying the specific symmetry within a given data distribution often proves ambiguous. To tackle this, some existing works learn symmetry in a data-driven manner, parameterizing and learning expected symmetry through data. However, these methods often rely on explicit knowledge, such as pre-defined Lie groups, which are typically restricted to linear or affine transformations. In this paper, we propose a novel symmetry learning algorithm based on transformations defined with one-parameter groups, continuously parameterized transformations flowing along the directions of vector fields called infinitesimal generators. Our method is built upon minimal inductive biases, encompassing not only commonly utilized symmetries rooted in Lie groups but also extending to symmetries derived from nonlinear generators. To learn these symmetries, we introduce a notion of a validity score that examine whether the transformed data is still valid for the given task. The validity score is designed to be fully differentiable and easily computable, enabling effective searches for transformations that achieve symmetries innate to the data. We apply our method mainly in two domains: image data and partial differential equations, and demonstrate its advantages. Our codes are available at \url{https://github.com/kogyeonghoon/learning-symmetry-from-scratch.git}.

对称性学习生成模型数据驱动向量场

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。