通过参数空间约束,让模型学会忽略偏见特征。
CosFairNet:A Parameter-Space based Approach for Bias Free Learning
- 在参数空间中同时训练偏见模型与去偏模型,强制其特征差异
- 在真实和合成数据集上,准确率与去偏效果均优于现有方法
- 适用于多种偏见类型和不同偏见比例的数据,适合鲁棒性要求高的场景
深度神经网络在有偏数据上训练时,常会无意中学到与标签强相关的偏见特征。现有方法要么预设偏见类型作为先验,要么重新加权样本以突出反偏样本,但都只在特征或样本空间间接处理偏见,无法控制权重传播,难以抑制偏见跨层扩散。本文提出一种新方法,直接在模型参数空间中干预,防止偏见传播。训练两个模型:一个用于建模偏见特征,另一个用于学习无偏细节,并受前者引导。在去偏模型的早期层强制与偏见模型相似,后期层则强制不相似,确保其学习无偏低层特征,而不继承偏见高层抽象。该方法在多种规模的合成与真实世界数据集上均提升了分类准确率与去偏效果,且对不同偏见类型及偏见样本占比具有鲁棒性。
原文摘要 · Abstract (English)
Deep neural networks trained on biased data often inadvertently learn unintended inference rules, particularly when labels are strongly correlated with biased features. Existing bias mitigation methods typically involve either a) predefining bias types and enforcing them as prior knowledge or b) reweighting training samples to emphasize bias-conflicting samples over bias-aligned samples. However, both strategies address bias indirectly in the feature or sample space, with no control over learned weights, making it difficult to control the bias propagation across different layers. Based on this observation, we introduce a novel approach to address bias directly in the model's parameter space, preventing its propagation across layers. Our method involves training two models: a bias model for biased features and a debias model for unbiased details, guided by the bias model. We enforce dissimilarity in the debias model's later layers and similarity in its initial layers with the bias model, ensuring it learns unbiased low-level features without adopting biased high-level abstractions. By incorporating this explicit constraint during training, our approach shows enhanced classification accuracy and debiasing effectiveness across various synthetic and real-world datasets of different sizes. Moreover, the proposed method demonstrates robustness across different bias types and percentages of biased samples in the training data. The code is available at: https://visdomlab.github.io/CosFairNet/
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。