提出理论框架,解释何时及如何用规范化解法提升模型泛化性能。
When and How to Canonize: A Generalization Perspective

- 通过覆盖数分析规范化的泛化误差边界
- 证明规范化解法效果取决于其规则性,可能优于或等同于不变架构
- 揭示希尔伯特曲线编码比字典序更优,适用于高维点云
尽管不变架构是处理对称数据的标准方法,但越来越多研究尝试通过群平均或规范化解法,将非不变主干网络转化为具有不变性的模型。然而,这些方法的泛化性质尚缺乏理论理解。本文提出一个理论框架,通过界定覆盖数来分析此类方法的泛化误差。我们建立严格的泛化层级:规范化解法的误差界至多等于结构不变或群平均模型的界,至少等同于非不变基线的界。进一步证明存在最优规范化解法可达到最优误差界,而糟糕的规范化解法则退化为非不变模型。针对点云处理中的置换群,我们严格证明字典序规范化的覆盖数随点云维度呈指数增长,而希尔伯特曲线规范化仅保证多项式增长。这为当前先进点云架构中希尔伯特曲线序列化的成功提供了首个形式化理论支持。实验验证了理论结论。代码已开源。
原文摘要 · Abstract (English)
While invariant architectures are standard for processing symmetric data, there is growing interest in achieving invariance by applying group averaging or canonization to non-invariant backbones. However, the theoretical generalization properties of these alternative strategies remain poorly understood. We introduce a theoretical framework to analyze the generalization error of these methods by bounding their covering numbers. We establish a rigorous generalization hierarchy: the error bounds of canonized models are at best equal to the error bounds of structurally invariant and group-averaged models, and at worst equal to the bounds of non-invariant baselines. Furthermore, we show that there exist optimal canonizations which attain the optimal error bounds, and poor canonizations which attain the non-invariant error bounds, and that this depends on the regularity of the canonization. Finally, applying this framework to permutation groups in point cloud processing, we rigorously prove that the covering number of lexicographical sorting grows exponentially with point cloud dimension, whereas Hilbert curve canonization guarantees polynomial growth. This provides the first formal theoretical justification for the empirical success of Hilbert curve serialization in state-of-the-art point cloud architectures. We conclude with experiments that support our theoretical claims. Code is available at https://github.com/yonatansverdlov/Canonization
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。