通过双分支投影消除模型偏差,提升少数群体表现。
Dual-Branch Cross-Projection Debiasing through Diffusion-based Disentanglement

- 用扩散模型分离语义概念,自动识别潜在偏见因素。
- 双分支结构分离目标与非因果特征,跨空间投影去偏。
- 仅微调0.22%参数,对少数群体准确率显著提升。
在存在偏见的数据集上训练的基础模型常依赖目标标签与非因果属性之间的虚假相关性,导致少数群体泛化能力差。偏见缓解面临两大挑战:其一,当组标签不可得时,现有无组监督方法通常从模型行为中隐式推断虚假属性,难以获得与现实偏见语义一致的因子;其二,即使有伪虚假监督,多数去偏方法采用单分支设计,在共享特征空间中目标与虚假属性仍固有纠缠。为此,我们提出置信度引导的偏见概念挖掘(CBCM),利用扩散-解耦的语义概念表示,无需属性标注即可识别可靠虚假属性。同时提出双分支交叉投影去偏(DCD),一种提示调优框架,将目标与虚假表征分置于两分支,通过跨空投影显式去除虚假信息,同时保留目标语义。在四个基准数据集上的大量实验表明,本方法在无组监督类方法中达到最优最差组准确率,且仅调优最多0.22%模型参数。源代码见补充材料。
原文摘要 · Abstract (English)
Foundation models trained on biased datasets often rely on spurious correlations between target labels and non-causal attributes, resulting in poor generalization on minority groups. Bias mitigation remains challenging due to two fundamental issues. First, when group labels are unavailable, existing group-unsupervised methods typically infer spurious attributes implicitly from model behavior, making it difficult to identify spurious factors that are semantically aligned with real-world biases. Second, even with pseudo spurious supervision, most existing debiasing methods follow a single-branch design that operates within a single shared feature space, where target and spurious attributes are intrinsically entangled. To address the first challenge, we introduce Confidence-guided Bias Concept Mining (CBCM), which leverages diffusion-disentangled, semantically grounded concept representations to identify reliable spurious attributes without attribute annotations. To address the second challenge, we propose Dual-branch Cross-projection Debiasing (DCD), a prompt-tuning framework that separates target and spurious representations into two branches and explicitly removes spurious information through cross null-space projection while preserving target-relevant semantics. Extensive experiments on four benchmark datasets show that our method achieves state-of-the-art worst group accuracy among group-unsupervised approaches, while tuning at most 0.22% of the model parameters. The source code is available in the supplementary materials.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。