根据输入敏感度选择性去偏，提升CLIP在零样本任务中的公平性与性能。
Selective Test-Time Debiasing for CLIP via Reward Gating
- 基于强化学习动态判断输入是否敏感，仅对敏感输入启用去偏
- 在公平性提升的同时，零样本性能不降反升
- 适合关注模型公平性的研究者与实际部署人员
视觉语言模型（VLM）展现出强大的零样本性能，但在涉及人物的查询中常延续社会刻板印象，导致人口分布偏差。现有去偏方法对所有输入统一施加修正，造成公平性与实用性的根本权衡：强去偏会扭曲无偏敏感输入的语义信息，弱去偏则无法有效缓解敏感输入的偏见。我们提出奖励门控测试时自适应（RG-TTA），一种基于强化学习的测试时自适应框架，根据输入的敏感度选择性应用去偏。RG-TTA在测试时策略适配过程中，基于每个输入的偏见敏感度自适应触发公平性正则化，同时仅对无偏敏感输入专注于优化跨模态对齐。在多个公平性基准（如FairFace、UTKFace）上的实验表明，该方法显著降低偏见，同时提升零样本性能，解决了统一去偏带来的权衡问题。
原文摘要 · Abstract (English)
Vision language models (VLMs) demonstrate strong zero-shot performance, but often perpetuate social stereotypes in person-centric queries, yielding skewed demographic distributions. Current debiasing methods apply uniform bias corrections across all input queries regardless of their bias sensitivity, creating a fundamental fairness--utility trade-off. Strong debiasing distorts semantically meaningful information in bias-insensitive queries, while weak debiasing fails to mitigate stereotypes in bias-sensitive ones. This one-size-fits-all approach hampers simultaneously achieving high utility on bias-insensitive queries and fairness on bias-sensitive queries. We introduce Reward-Gated Test-Time Adaptation (RG-TTA), a reinforcement learning-based test-time adaptation framework that selectively applies debiasing based on input sensitivity. RG-TTA adaptively triggers fairness regularization based on the bias sensitivity of each input during test-time policy adaptation, while focusing exclusively on optimizing cross-modal alignment for bias-insensitive inputs. Experiments on fairness benchmarks (e.g., FairFace, UTKFace) demonstrate substantial bias reduction while simultaneously improving zero-shot utility, resolving the trade-off of uniform debiasing.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。