提出视觉语言模型安全校准新基准，解决答错与拒答的双重问题。
VSCBench: Bridging the Gap in Vision-Language Model Safety Calibration
- 构建3600对图像文本对，区分安全与非安全场景。
- 11个主流模型均存在过度安全或安全不足问题。
- 揭示校准与模型实用性间的权衡，推动更优方法发展。
视觉语言模型（VLMs）的安全对齐受到广泛关注，但现有方法主要关注模型对危险提问的回应（欠安全），而忽视了对安全提问的拒绝（过安全）。本文提出“安全校准”概念，系统解决两类问题。我们构建了名为 VSCBench 的新数据集，包含3,600张图像-文本对，这些对在视觉或文本上相似，但在安全性上存在差异，适用于评估图像主导和文本主导场景下的安全校准效果。基于该基准，我们评估了11个广泛使用的VLMs，发现两者均存在严重问题。进一步研究四种改进方法，发现虽能缓解安全问题，但会损害模型实用性。这一权衡凸显了开发先进校准方法的紧迫性。我们的代码与数据已开源。
原文摘要 · Abstract (English)
The rapid advancement of vision-language models (VLMs) has brought a lot of attention to their safety alignment. However, existing methods have primarily focused on model undersafety, where the model responds to hazardous queries, while neglecting oversafety, where the model refuses to answer safe queries. In this paper, we introduce the concept of $\textit{safety calibration}$, which systematically addresses both undersafety and oversafety. Specifically, we present $\textbf{VSCBench}$, a novel dataset of 3,600 image-text pairs that are visually or textually similar but differ in terms of safety, which is designed to evaluate safety calibration across image-centric and text-centric scenarios. Based on our benchmark, we evaluate safety calibration across eleven widely used VLMs. Our extensive experiments revealed major issues with both undersafety and oversafety. We further investigated four approaches to improve the model's safety calibration. We found that even though some methods effectively calibrated the models' safety problems, these methods also lead to the degradation of models' utility. This trade-off underscores the urgent need for advanced calibration methods, and our benchmark provides a valuable tool for evaluating future approaches. Our code and data are available at https://github.com/jiahuigeng/VSCBench.git.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。