分离出模型在分布外失效中的捷径依赖、规则切换与跨族失败
Separating Shortcut Transition from Cross-Family OOD Failure in a Minimal Model
- 构建最小二元模型，区分不变特征与家族特有捷径特征
- 当训练捷径信号超过不变信号时，分类器会切换至捷径规则
- 同一训练过渡可能导致不同测试族出现过拟合或错误率上升
捷径特征常被用来解释分布外（OOD）失败，但训练相关性、学习到的捷径使用与测试时的失败并不必然一致。我们研究了一个包含一个不变坐标和一个家族依赖捷径坐标的最小二元模型。在确定性情况下，正向平均捷径相关性会将逻辑回归ERM拉向正的捷径权重，但岭正则化可保持分类器以不变特征为主导，防止确定性OOD失败。当不变坐标存在噪声时，一旦训练阶段的捷径信号超过不变信号，岭-逻辑回归ERM就会切换到捷径规则。这种切换是否导致失败取决于保留测试族：较弱的捷径相关性导致正的额外风险，符号翻转的测试族产生高于随机水平的错误率。合成实验验证了这些分析结果，并表明相同的训练过渡可能在不同保留族上引发不同后果。该模型清晰分离了捷径吸引、捷径规则切换与跨族OOD失败三个过程。
原文摘要 · Abstract (English)
Shortcut features are often invoked to explain out-of-distribution (OOD) failure, but training correlation, learned shortcut use, and test-time failure need not coincide. We study a minimal binary model with one invariant coordinate and one family-dependent shortcut coordinate. In the deterministic regime, positive average shortcut correlation pulls logistic ERM toward positive shortcut weight, but ridge regularization keeps the classifier invariant-dominated and prevents deterministic OOD failure. When the invariant coordinate is noisy, ridge-logistic ERM switches to the shortcut rule once the training shortcut signal exceeds the invariant signal. Whether that transition causes failure depends on the held-out family: weaker shortcut correlation yields positive excess risk, and sign-flipped families yield above-chance error. Synthetic checks match these analytic regimes and show that the same training-side transition can have different held-out consequences. The model separates shortcut attraction, shortcut-rule transition, and cross-family OOD failure.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。