研究马尔可夫数据下等变网络的泛化能力，给出可落地的优化方向。
Generalization Bounds for Equivariant Networks on Markov Data
- 用改进的McDiarmid不等式结合径向复杂度分析马尔可夫数据泛化性
- 推导出等变网络在固定宽度下的泛化上界，明确低维不可约表示更优
- 为具有依赖结构的数据设计等变网络提供理论依据，适合对齐理论研究者
等变神经网络在具有对称性的复杂数据结构中起关键作用。然而，将等变性与马尔可夫特性结合面临挑战，因这类数据存在内在依赖。以往研究多基于独立同分布假设，忽略马尔可夫依赖的影响。本文通过引入新的McDiarmid不等式，利用径向复杂度作为模型容量的核心度量，推导出在马尔可夫数据上训练的神经网络的泛化界。进一步借助群论计算等变约束下的覆盖数，从而得到基于该覆盖数的径向复杂度上界。该上界为选择低维不可约表示提供了实用指导，有助于提升固定宽度等变网络的泛化性能。
原文摘要 · Abstract (English)
Equivariant neural networks play a pivotal role in analyzing datasets with symmetry properties, particularly in complex data structures. However, integrating equivariance with Markov properties presents notable challenges due to the inherent dependencies within such data. Previous research has primarily concentrated on establishing generalization bounds under the assumption of independently and identically distributed data, frequently neglecting the influence of Markov dependencies. In this study, we investigate the impact of Markov properties on generalization performance alongside the role of equivariance within this context. We begin by applying a new McDiarmid's inequality to derive a generalization bound for neural networks trained on Markov datasets, using Rademacher complexity as a central measure of model capacity. Subsequently, we utilize group theory to compute the covering number under equivariant constraints, enabling us to obtain an upper bound on the Rademacher complexity based on this covering number. This bound provides practical insights into selecting low-dimensional irreducible representations, enhancing generalization performance for fixed-width equivariant neural networks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。