arXiv:2606.17961cs.CVcs.AI2026-06

分析相似性位置编码在旋转下的鲁棒性,证明其理论稳定并实证优于传统编码。

Robustness of Similarity-based Positional Encoding Under Rotations: Theoretical Analysis and Experimental Validation

论文配图:Robustness of Similarity-based Positional Encoding Under Rotations: Theoretical Analysis and Experimental Validation
图 1 · 摘自论文原文
  • 通过配对关系建模位置,结合李普希茨假设推导旋转扰动的稳定界。
  • 在四个数据集上,旋转条件下准确率、F1等指标显著优于标准位置编码。
  • 适合关注几何不变性的视觉任务,如医学影像分类与旋转敏感场景。

位置编码是Transformer架构中的核心组件,用于注入输入的空间或顺序信息。近年来,基于相似性的位置编码(simPE)作为一种灵活框架,通过成对关系表示位置结构,已在医学影像领域展现潜力——该领域中图像采集时易出现微小旋转,源于设备、体位或轻微错位。尽管实证表现良好,但simPE在几何扰动下的理论行为尚未被充分刻画。本文结合理论分析与实验验证,研究simPE在旋转下的鲁棒性。首先证明simPE通常不具备旋转不变性;随后在元素组件满足弱李普希茨条件的前提下,证明其在旋转扰动下具有稳定性,并推导出弗罗比尼乌斯范数下的显式扰动界。实验在四个受控数据集上验证:合成箭头数据集、合成形状数据集(四类几何形状)、合成数字数据集及基准图像分类数据集FashionMNIST。训练与验证图像保持固定朝向,测试图像施加递增旋转角度。所有数据集均显示,simPE在小到中等旋转角度下,准确率、F1分数、精确率和召回率持续优于标准学习型位置编码,验证了理论稳定性保证。

原文摘要 · Abstract (English)

Positional encoding is a fundamental component of Transformer architectures, as it injects information about the spatial or sequential arrangement of inputs. Among recent alternatives to standard absolute and sinusoidal encodings, similarity-based positional encoding (simPE) has emerged as a flexible framework for representing positional structure through pairwise relations. simPE was originally designed for medical imaging applications, where geometric robustness is especially relevant: small rotations naturally arise during image acquisition, induced by imaging instruments, patient positioning, or slight acquisition misalignments. Despite its empirical promise, the theoretical behavior of simPE under geometric perturbations has not been fully characterized. In this paper, we study the robustness of simPE with respect to rotations, combining formal theoretical analysis with experimental validation. We first show that simPE is generally not rotation-invariant. We then prove that, under mild Lipschitz assumptions on the elementary components, simPE is stable under rotational perturbations and derive explicit perturbation bounds in Frobenius norm. We validate these findings experimentally on four controlled datasets--a synthetic Arrow dataset, a synthetic Shapes dataset (four geometric shape categories), a synthetic Digits dataset, and a benchmark image classification dataset (FashionMNIST)--in which training and validation images are kept in a fixed canonical orientation while test images are subjected to increasing rotation angles. Across all datasets, simPE consistently outperforms standard learned positional encoding in terms of accuracy, F1 score, precision, and recall under rotation, particularly in the small-to-moderate angle regime, corroborating the theoretical stability guarantees.

位置编码旋转鲁棒性Transformer理论分析

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。