提出一种动态纠偏方法，让生成式视觉语言模型输出更公平。
GGSS: Geodesic-Gated Spherical Steering for Inference-Time Debiasing of Generative Vision-Language Models
- 在单位超球面上发现反事实偏差子空间，沿测地线调整视觉标记。
- 在四种模型上平均偏差最低，且保持图像理解能力仅下降0.6个百分点。
- 适合需要公平性保障的生成式视觉语言模型应用，如内容生成与评测。
生成式视觉语言模型（VLMs）广泛应用于人机交互场景，但即使图像仅在感知种族或性别等属性上存在微小差异，仍可能产生人口统计学偏差。现有推理时纠偏方法多针对静态嵌入或CLIP类模型，难以适配生成式VLMs。本文提出GGSS——测地线门控球面引导，一种保范干预方法：在单位超球面上发现反事实偏差子空间，沿测地线弧调整视觉标记，并通过自适应门控聚焦于携带更强人口统计信号的标记。我们在四个生成式VLMs上评估了十种改进的推理时纠偏基线及提示法，在单一运行点协议下覆盖类别、成对及职业-性别偏差测试，并衡量通用视觉语言能力。结果表明，GGSS在所有四款模型上平均偏差最低，其中三款在成对置换检验中显著更低，同时保持MMStar准确率与原始基线相差不超过±0.6个百分点。代码已开源：https://github.com/dukesun99/GGSS。
原文摘要 · Abstract (English)
Generative vision-language models (VLMs) are increasingly used in human-centered settings, yet they can produce demographically biased outputs even when images differ only in controlled attributes such as perceived race or gender. However, existing inference-time debiasers were largely designed for static embeddings or CLIP-like models rather than generative VLMs. We propose GGSS---Geodesic-Gated Spherical Steering---a norm-preserving intervention that discovers a counterfactual bias subspace on the unit hypersphere, steers visual tokens along geodesic arcs, and uses an adaptive gate to focus correction on tokens that carry stronger demographic signal. We evaluate four generative VLMs against ten adapted inference-time debiasing baselines and prompt-based mitigation under a single operating-point protocol across categorical, pairwise, and occupation-gender bias tests, while also measuring general visual-language capability. GGSS achieves the lowest average bias on all four models, significant on three of four backbones under paired permutation tests, while preserving MMStar accuracy within +/- 0.6 p.p. of the unsteered baseline. Code is available at https://github.com/dukesun99/GGSS.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。