提出GMBM框架,一站式解决视觉模型中的多重偏见问题。
Multi Attribute Bias Mitigation via Representation Learning
- 分两阶段训练:先识别各属性偏见,再通过梯度抑制消除这些偏见。
- 在多个数据集上将最差组准确率提升,多属性偏见放大减半。
- 新指标SBA能有效区分模型偏见与分布差异,适合复杂场景下的公平性评估。
现实图像常存在纹理、水印、性别化妆容、场景物体组合等多种重叠偏见,共同损害现代视觉模型的鲁棒性与公平性。单独处理每种偏见效果有限,因缓解一种偏见可能加剧其他偏见。本文提出通用多偏见缓解(GMBM)框架,仅需训练时的分组标签,测试时即实现偏见最小化。第一阶段,自适应偏见融合学习(ABIL)为每个属性训练编码器并融合至主干网络,迫使分类器显式识别这些捷径;第二阶段,梯度抑制微调从主干梯度中剔除这些偏见方向,得到一个紧凑且忽略所有捷径的网络。此外,发现现有偏见度量在子群不平衡和训练测试分布漂移下失效,因此引入缩放偏见放大(SBA):一种测试时度量,可分离模型引起的偏见放大与分布差异。在FB CMNIST、CelebA和COCO上验证,GMBM显著提升最差组准确率,多属性偏见放大降低50%,并在偏见复杂度和分布漂移增强时仍保持最低SBA值,成为首个实用的端到端多偏见视觉识别解决方案。
原文摘要 · Abstract (English)
Real world images frequently exhibit multiple overlapping biases, including textures, watermarks, gendered makeup, scene object pairings, etc. These biases collectively impair the performance of modern vision models, undermining both their robustness and fairness. Addressing these biases individually proves inadequate, as mitigating one bias often permits or intensifies others. We tackle this multi bias problem with Generalized Multi Bias Mitigation (GMBM), a lean two stage framework that needs group labels only while training and minimizes bias at test time. First, Adaptive Bias Integrated Learning (ABIL) deliberately identifies the influence of known shortcuts by training encoders for each attribute and integrating them with the main backbone, compelling the classifier to explicitly recognize these biases. Then Gradient Suppression Fine Tuning prunes those very bias directions from the backbone's gradients, leaving a single compact network that ignores all the shortcuts it just learned to recognize. Moreover we find that existing bias metrics break under subgroup imbalance and train test distribution shifts, so we introduce Scaled Bias Amplification (SBA): a test time measure that disentangles model induced bias amplification from distributional differences. We validate GMBM on FB CMNIST, CelebA, and COCO, where we boost worst group accuracy, halve multi attribute bias amplification, and set a new low in SBA even as bias complexity and distribution shifts intensify, making GMBM the first practical, end to end multibias solution for visual recognition. Project page: http://visdomlab.github.io/GMBM/
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。