证明近似对称比精确对称更容易实现,且复杂度呈指数级差异。
Achieving Approximate Symmetry Is Exponentially Easier than Exact Symmetry
- 用平均复杂度衡量对称性约束成本,提出理论分析框架。
- 精确对称需线性复杂度,近似对称仅需对数复杂度。
- 为实际应用中偏好近似对称提供了首个理论依据。
在机器学习模型中强制精确对称性常带来科学应用的显著提升,作为强大的归纳偏置。然而,近期研究指出,采用近似对称可提供更大灵活性与鲁棒性。尽管有令人信服的实证证据,理论理解仍不足,尤其是精确与近似对称之间的直接比较缺失。本文首次提出平均复杂度框架,量化通过平均实现对称性的代价。主要结果为:在标准条件下,精确对称需线性平均复杂度,而近似对称仅需对数复杂度(关于群大小)。这是首个此类理论分离,正式支持实践中近似对称的优越性。此外,本文工具对机器学习中对称性的更广泛研究亦具独立价值。
原文摘要 · Abstract (English)
Enforcing exact symmetry in machine learning models often yields significant gains in scientific applications, serving as a powerful inductive bias. However, recent work suggests that relying on approximate symmetry can offer greater flexibility and robustness. Despite promising empirical evidence, there has been little theoretical understanding, and in particular, a direct comparison between exact and approximate symmetry is missing from the literature. In this paper, we initiate this study by asking: What is the cost of enforcing exact versus approximate symmetry? To address this question, we introduce averaging complexity, a framework for quantifying the cost of enforcing symmetry via averaging. Our main result is an exponential separation: under standard conditions, exact symmetry requires linear averaging complexity, whereas approximate symmetry can be attained with only logarithmic complexity in the group size. To the best of our knowledge, this provides the first theoretical separation of these two cases, formally justifying why approximate symmetry may be preferable in practice. Beyond this, our tools and techniques may be of independent interest for the broader study of symmetries in machine learning.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。