通过数据增强实现贝叶斯神经网络的精确等变性，提升模型性能与对称性保持能力。
Equivariance and Augmentation for Bayesian Neural Networks
- 利用变分推断下的贝叶斯神经网络，结合数据增强实现等变性
- 提出轨道扩展等新方法，显著降低等变误差并提升整体性能
- 适用于需保持对称性的科学计算与医学图像任务
对称性在众多深度学习任务中至关重要，涵盖科学计算到医学影像。当前争议在于应通过网络结构施加对称约束（等变网络）还是通过增强数据学习对称性。尽管等变网络理论成熟，但数据增强研究较少，因需控制训练动态。受最新发现启发——增强的无限深度集成恰好是等变的，本文研究基于变分推断的贝叶斯神经网络中的数据增强。聚焦指数族变分分布，推导出达到精确等变性的条件，并给出等变误差的上界。进一步提出三种新颖的对称化技术，显著增强数据增强效果。大量数值实验表明，其中一种方法（轨道扩展）在等变性和整体性能上均优于基线。代码已开源于 github.com/dmw1998/augment-BNNs。
原文摘要 · Abstract (English)
Symmetries are important for many deep learning tasks, ranging from applications in the sciences to medical imaging. However, there is an ongoing debate about whether to impose symmetry constraints on the neural network architecture (yielding equivariant neural networks) or learn them from augmented training data. Although equivariant networks are well-studied theoretically, much less is known about data augmentation, since analyzing augmentation requires control over the training dynamics. Inspired by recent results that show that augmented infinite deep ensembles are exactly equivariant, we study data augmentation for Bayesian neural networks (BNNs) trained with variational inference. We focus on variational distributions in the exponential family and derive conditions under which exact equivariance is reached. We furthermore obtain bounds on the equivariance error and introduce three novel symmetrization techniques which boost the effect of data augmentation in this setting. We conduct extensive numerical experiments which show that one of our symmetrization methods (orbit expansion) outperforms the baseline in both equivariance and overall performance. Our code is available at github.com/dmw1998/augment-BNNs
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。