arXiv:2604.21395cs.LGcs.AI2026-04

监督学习忽略隐变量扰动,导致模型对无关特征敏感。

Supervised Learning Has a Geometric Blind Spot

  • 提出几何盲点概念:模型训练后仍对无关输入扰动敏感。
  • 实验证明视觉变压器在无监督轴时,等距编码匹配表现最优。
  • 建议报告分类布局几何,尤其当干扰轴未知时优先使用等距匹配。

常规监督训练仅最小化任务损失即停止,忽视输入沿助于拟合标签但部署时为干扰的方向扰动所引发的表征偏移——这种残留敏感性称为经验风险最小化的几何盲点。在高斯线性模型中,若干扰影响标签条件且解码器有有限利普希茨常数,总体均方误差对线性化表征漂移设定了下限。该区分也预测了对抗训练的失效模式:雅可比范数可能下降,而干净类别几何却恶化。通过类别布局得分追踪此脱节现象,并在干扰轴未知时采用等距编码匹配(在任务损失约束下惩罚phi(x)与phi(x+delta)的平方距离)。在从头训练的CIFAR-10 Vision Transformer上,投影梯度下降获得最小的雅可比弗罗贝尼乌斯范数,但清洁布局得分最差(1.353±0.020,三组种子),高于仅任务训练(1.093);等距匹配取得最佳得分(0.904)。线性高斯情形下的漂移下界已证明;深度网络与跨任务排序为协议实证。设计建议:除任务得分外,应报告类别布局几何;当干扰轴未知时,优选等距编码匹配。

原文摘要 · Abstract (English)

Ordinary supervised training minimises the task loss and then stops. It never pays for how far the representation moves when the input is nudged along directions that helped fit training labels---including directions that are nuisance at deployment. We call that leftover sensitivity the geometric blind spot of empirical risk minimisation. In a Gaussian linear model where the nuisance enters the label conditional and the decoder has finite Lipschitz constant, population MSE forces a floor on linearised representation drift. The same distinction predicts a failure mode of adversarial training: Jacobian magnitude can fall while clean class geometry worsens. We track that dissociation with a class-layout score and study isotropic encoder matching---penalising the squared distance between phi(x) and phi(x+delta) for Gaussian delta under a task-loss cap---when nuisance axes are unknown. On a Vision Transformer trained from scratch on CIFAR-10, projected gradient descent attains the smallest Jacobian Frobenius yet the worst clean layout score (1.353+/-0.020 over three seeds), above task-only training (1.093); isotropic matching attains the best (0.904). The drift floor is proved for the linear-Gaussian case; deep nets and cross-task orderings are protocol empirics. Design rule: report class-layout geometry beside the task score; prefer isotropic encoder matching when axes are unknown.

监督学习几何盲点表征鲁棒性视觉变换器

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。