用公平性正则化改进视觉模型的低秩微调，减少群体间性能差异。
FairLoRA: Unpacking Bias Mitigation in Vision Models with Fairness-Driven Low-Rank Adaptation
- 在LoRA中引入公平性正则项，通过最小化每类损失方差来降低偏差。
- 实验表明高秩并非通用解法，模型、数据和任务影响偏见缓解效果。
- 强调需多指标评估公平性，避免仅依赖训练优化的单一指标。
近年来，参数高效微调方法如低秩适配（LoRA）因其能以较低计算成本实现与全量微调相当的性能，受到广泛关注。为系统解决大语言模型中的公平性问题，先前研究通常使用比常规更高的LoRA秩对公平性特定数据进行微调。本文提出FairLoRA，一种针对公平性的新型LoRA正则化方法，旨在通过最小化每类损失方差来减少不同数据子群间的性能差异。据我们所知，这是首个通过LoRA实现公平性微调的工作。实验结果表明，提高LoRA秩以缓解偏见并非普遍适用，其效果取决于预训练模型、数据集和任务。我们在包含ViT、DiNO和CLIP等多种视觉模型上，系统评估了FairLoRA在分布偏移场景下的表现。此外，强调必须采用多种公平性度量才能全面评估公平性，而非仅依赖训练中优化的指标。
原文摘要 · Abstract (English)
Recent advances in parameter-efficient fine-tuning methods, such as Low Rank Adaptation (LoRA), have gained significant attention for their ability to efficiently adapt large foundational models to various downstream tasks. These methods are appreciated for achieving performance comparable to full fine-tuning on aggregate-level metrics, while significantly reducing computational costs. To systematically address fairness in LLMs previous studies fine-tune on fairness specific data using a larger LoRA rank than typically used. In this paper, we introduce FairLoRA, a novel fairness-specific regularizer for LoRA aimed at reducing performance disparities across data subgroups by minimizing per-class variance in loss. To the best of our knowledge, we are the first to introduce a fairness based finetuning through LoRA. Our results demonstrate that the need for higher ranks to mitigate bias is not universal; it depends on factors such as the pre-trained model, dataset, and task. More importantly, we systematically evaluate FairLoRA across various vision models, including ViT, DiNO, and CLIP, in scenarios involving distribution shifts. We further emphasize the necessity of using multiple fairness metrics to obtain a holistic assessment of fairness, rather than relying solely on the metric optimized during training.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。