提出可同时处理平移与连续旋转的等变Transformer，提升模型鲁棒性。
Harmformer: Harmonic Networks Meet Transformers for Continuous Roto-Translation Equivariance
- 用谐波函数构建连续旋转等变的Transformer，结合卷积主干
- 在无旋转训练样本下仍保持稳定，准确率优于现有等变模型
- 适合需要高旋转鲁棒性的视觉任务，如医学图像分析
CNNs 具有固有的平移等变性，带来参数和数据使用效率高、学习速度快、鲁棒性强等优势。该概念已被成功扩展至离散旋转群的群卷积以及涵盖360°的连续旋转群的谐波函数。本文探讨自注意力机制与完整旋转等变性的兼容性，不同于以往仅关注离散旋转的研究。我们提出 Harmformer，一种具有卷积主干的谐波Transformer，实现平移与连续旋转的双重等变性。伴随端到端等变性证明，Harmformer 不仅超越了先前的等变Transformer，且在未经旋转样本训练的情况下，仍对任意连续旋转保持内在稳定性。
原文摘要 · Abstract (English)
CNNs exhibit inherent equivariance to image translation, leading to efficient parameter and data usage, faster learning, and improved robustness. The concept of translation equivariant networks has been successfully extended to rotation transformation using group convolution for discrete rotation groups and harmonic functions for the continuous rotation group encompassing $360^\circ$. We explore the compatibility of the SA mechanism with full rotation equivariance, in contrast to previous studies that focused on discrete rotation. We introduce the Harmformer, a harmonic transformer with a convolutional stem that achieves equivariance for both translation and continuous rotation. Accompanied by an end-to-end equivariance proof, the Harmformer not only outperforms previous equivariant transformers, but also demonstrates inherent stability under any continuous rotation, even without seeing rotated samples during training.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。