arXiv:2507.17001cs.LG2025-07

提出利用偏差提升分布外泛化能力的新框架,打破消除偏差的传统思路。

Should Bias be Eliminated? A General Framework to Use Bias for OOD Generalization

  • 用生成模型识别数据生成中的潜在偏差因子
  • 构建环境感知的偏差感知预测器,在多个场景下表现更优
  • 适合需要鲁棒性与自适应能力的现实分布外任务

大多数分布外(OOD)泛化方法通过消除上下文偏差来学习域不变表示。本文提出关键问题:偏差是否应被消除?若否,能否在一般情况下有效利用偏差?我们首先进行理论分析,揭示偏差在特定条件下可带来正向作用。尽管理论上偏差可能有益,但其有害与有益成分常纠缠难分。现有方法依赖不变特征的可靠预测来修正偏差,但在真实世界中假设过强,尤其当目标域从训练到测试发生偏移时。为此,我们提出一个更通用的偏差利用框架:采用生成模型捕捉数据生成过程,识别潜在偏差因素,并据此构建偏差感知预测器。由于该预测器在不同环境中可能变化,我们先估计环境状态,训练各环境下的预测器,再以混合专家方式融合为最终预测;同时构建一个通用不变预测器,可在标签偏移下保持不变,引导偏差感知预测器的自适应调整。在合成数据和标准域泛化基准上的实验表明,该方法始终优于仅依赖不变性的基线、近期偏差利用方法及先进基线,显著提升鲁棒性与适应性。

原文摘要 · Abstract (English)

Most approaches to out-of-distribution (OOD) generalization learn domain-invariant representations by discarding contextual bias. In this paper, we raise a critical question: Should bias be eliminated? If not, is there a general way to leverage bias for better OOD generalization? To answer these questions, we first provide a theoretical analysis that characterizes the circumstances in which biased features contribute positively. Although theoretical results show that bias may sometimes play a positive role, leveraging it effectively is non-trivial, since its harmful and beneficial components are often entangled. Recent advances have sought to refine the prediction of bias by presuming reliable predictions from invariant features. However, such assumptions may be too strong in the real world, especially when the target also shifts from training to testing domains. Motivated by this challenge, we introduce a framework to leverage bias in a more general scenario. Specifically, we employ a generative model to capture the data generation process and identify the underlying bias factors, which are then used to construct a bias-aware predictor. Since the bias-aware predictor may shift across environments, we first estimate the environment state to train predictors under different environments, combining them as a mixture of domain experts for the final prediction. Then, we build a general invariant predictor, which can be invariant under label shift to guide the adaptation of the bias-aware predictor. Evaluations on synthetic data and standard domain generalization benchmarks demonstrate that our method consistently outperforms both invariance only baselines, recent bias utilization approaches and advanced baselines, yielding improved robustness and adaptability.

分布外泛化偏差利用生成模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。