arXiv:2603.06693cs.CVcs.LG2026-03被引 1

提出分层解耦的软等变正则化,提升自监督学习在几何扰动下的鲁棒性。

Soft Equivariance Regularization for Invariant Self-Supervised Learning

  • 将不变性与等变性在不同层级分别施加,中间特征图用解析群作用实现软等变正则化
  • 在ImageNet-1k上线性评估提升0.84%准确率,且在图像噪声/遮挡测试中提升1.11%~1.22%
  • 无需额外预测头或变换标签,仅增加1.008倍训练计算量,适合大模型高效改进

自监督学习通常学习对语义保持的增强具有不变性的表征。虽然这对识别任务有效,但强制强不变性会抑制对几何扰动鲁棒性和空间敏感迁移有用的变换相关结构。现有方法虽引入等变目标,但常在同一最终表示上施加,存在权衡:将等变正则化推向深层可提升等变分数,却降低ImageNet-1k线性评估性能。为此,我们提出软等变正则化(SER),一种插件式正则器,解耦不变性与等变性的施加位置:保留最终嵌入的原始SSL目标,同时在中间空间标记图上通过解析定义的群作用ρ_g直接在特征空间中软性鼓励等变性。SER不学习/预测样本级变换代码或标签,无需辅助变换预测头,仅增加1.008倍训练浮点运算量。在ImageNet-1k ViT-S/16预训练中,SER使MoCo-v3在线性评估中提升+0.84% Top-1,在严格匹配双视图设置下表现优于对比方法;在相同视图数下,其在所比较的不变性+等变性附加模块中达到最高图像分类准确率。此外,它在ImageNet-C/P上提升+1.11/+1.22 Top-1,冻结主干的COCO检测提升+1.7 mAP。将相同分层解耦策略应用于现有基线也提升性能,表明该设计原则具普适性。

原文摘要 · Abstract (English)

Self-supervised learning (SSL) typically learns representations invariant to semantic-preserving augmentations. While effective for recognition, enforcing strong invariance can suppress transformation-dependent structure that is useful for robustness to geometric perturbations and spatially sensitive transfer. A growing body of work, therefore, augments invariance-based SSL with equivariance objectives, but these objectives are often imposed on the same final representation. We empirically observe a trade-off in this coupled setting: pushing equivariance regularization toward deeper layers improves equivariance scores but degrades ImageNet-1k linear evaluation, motivating a layer-decoupled design. Motivated by this trade-off, we propose Soft Equivariance Regularization (SER), a plug-in regularizer that decouples where invariance and equivariance are enforced: we keep the base SSL objective unchanged on the final embedding, while softly encouraging equivariance on an intermediate spatial token map via analytically specified group actions $ρ_g$ applied directly in feature space. SER learns/predicts no per-sample transformation codes/labels, requires no auxiliary transformation-prediction head, and adds only 1.008x training FLOPs. On ImageNet-1k ViT-S/16 pretraining, SER improves MoCo-v3 by +0.84 Top-1 in linear evaluation under a strictly matched 2-view setting and consistently improves DINO and Barlow Twins; under matched view counts, SER achieves the best ImageNet-1k linear-eval Top-1 among the compared invariance+equivariance add-ons. SER further improves ImageNet-C/P by +1.11/+1.22 Top-1 and frozen-backbone COCO detection by +1.7 mAP. Finally, applying the same layer-decoupling recipe to existing invariance+equivariance baselinesimproves their accuracy, suggesting layer decoupling as a general design principle for combining invariance and equivariance.

自监督学习等变正则化模型鲁棒性视觉表征

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。