arXiv:2512.05931cs.LGstat.ML2025-12

现有分歧差异代理损失不贝叶斯一致,新方法可有效提升鲁棒性评估精度

On the Bayes Inconsistency of Disagreement Discrepancy Surrogates

  • 提出新型分歧损失,结合交叉熵实现贝叶斯一致的代理优化
  • 理论证明现有代理损失存在最优差距,最大可达15%误差
  • 在对抗条件下显著优于基准方法,适合高可靠性场景

深度神经网络在真实场景中常因分布偏移而失效,成为构建安全可靠系统的关键障碍。一种新兴方法依赖于‘分歧差异’——衡量两模型在分布变化下的分歧变化程度。最大化该度量已用于估计分布偏移下的误差、检测有害偏移及训练更鲁棒模型。然而,此优化涉及不可导的0-1损失,需使用实际代理损失。我们证明现有分歧差异代理损失均非贝叶斯一致,揭示根本缺陷:最大化这些代理损失可能无法真正最大化真实分歧差异。为此,我们给出新的理论结果,提供此类代理损失的最优差距上下界。基于该理论,提出一种新型分歧损失,与交叉熵结合后,可保证贝叶斯一致性。在多种基准上的实证评估表明,本方法在挑战性对抗条件下比现有方法提供更准确、更鲁棒的分歧差异估计。

原文摘要 · Abstract (English)

Deep neural networks often fail when deployed in real-world contexts due to distribution shift, a critical barrier to building safe and reliable systems. An emerging approach to address this problem relies on \emph{disagreement discrepancy} -- a measure of how the disagreement between two models changes under a shifting distribution. The process of maximizing this measure has seen applications in bounding error under shifts, testing for harmful shifts, and training more robust models. However, this optimization involves the non-differentiable zero-one loss, necessitating the use of practical surrogate losses. We prove that existing surrogates for disagreement discrepancy are not Bayes consistent, revealing a fundamental flaw: maximizing these surrogates can fail to maximize the true disagreement discrepancy. To address this, we introduce new theoretical results providing both upper and lower bounds on the optimality gap for such surrogates. Guided by this theory, we propose a novel disagreement loss that, when paired with cross-entropy, yields a provably consistent surrogate for disagreement discrepancy. Empirical evaluations across diverse benchmarks demonstrate that our method provides more accurate and robust estimates of disagreement discrepancy than existing approaches, particularly under challenging adversarial conditions.

分布偏移鲁棒性代理损失贝叶斯一致

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。