从普通模型中提取无偏子网络,无需重训或额外数据。
Bias In, Bias Out? Finding Unbiased Subnetworks in Vanilla Models
- 通过剪枝技术在训练好的模型中找无偏子网络
- 提取的子网络减少对偏见特征依赖,性能稳定
- 适合想低成本去偏的开发者和研究者
深度学习中的算法偏见问题催生了多种去偏方法,大多依赖复杂的训练过程或数据操作。本文提出一种新策略——偏差不变子网络提取(BISE),旨在从常规训练的模型中直接提取出‘无偏’子网络,无需重新训练或微调原始参数。实验表明,仅通过剪枝即可实现该目标,所提取的子网络无需修改即可运行,显著降低对偏见特征的依赖,同时保持稳健性能。该方法通过移除参数实现结构化去偏,相比依赖数据或全量重训的方案更高效。在多个主流基准测试中,结果验证了其在性能与计算效率上的优势。
原文摘要 · Abstract (English)
The issue of algorithmic biases in deep learning has led to the development of various debiasing techniques, many of which perform complex training procedures or dataset manipulation. However, an intriguing question arises: is it possible to extract fair and bias-agnostic subnetworks from standard vanilla-trained models without relying on additional data, such as unbiased training set? In this work, we introduce Bias-Invariant Subnetwork Extraction (BISE), a learning strategy that identifies and isolates "bias-free" subnetworks that already exist within conventionally trained models, without retraining or finetuning the original parameters. Our approach demonstrates that such subnetworks can be extracted via pruning and can operate without modification, effectively relying less on biased features and maintaining robust performance. Our findings contribute towards efficient bias mitigation through structural adaptation of pre-trained neural networks via parameter removal, as opposed to costly strategies that are either data-centric or involve (re)training all model parameters. Extensive experiments on common benchmarks show the advantages of our approach in terms of the performance and computational efficiency of the resulting debiased model.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。