研究微调如何影响视觉语言模型预测的合理性，发现正确答案未必可靠。
Beyond Accuracy: On the Effects of Fine-tuning Towards Vision-Language Model's Prediction Rationality
- 提出可信度与推理可靠性新指标，量化模型决策依据是否合理。
- 微调后模型常凭无效证据做出正确判断，看似对实则不可信。
- 在分布外数据和不同场景下结果一致，适合安全关键领域研究者参考。
视觉语言模型（如CLIP）已广泛应用。研究人员在安全关键领域积极对其进行微调。在此类场景中，预测合理性至关重要：预测应正确且基于有效证据。然而，现有研究极少探讨微调对预测合理性的影响。为此，我们提出了两个新指标——预测可信度与推理可靠性，并在多种设置下开展广泛实验，观察到一些有趣现象：一方面，常用的微调方法虽提升了正确率，但模型更多依赖无效证据进行判断，可能削弱正确预测的可信度；另一方面，在识别出目标对象有效证据后，微调后的模型更可能做出正确预测。这些发现也在分布外数据和不同实验设置下保持一致。本研究旨在为视觉语言模型的微调提供新的视角。
原文摘要 · Abstract (English)
Vision-Language Models (VLMs), such as CLIP, have already seen widespread applications. Researchers actively engage in further fine-tuning VLMs in safety-critical domains. In these domains, prediction rationality is crucial: the prediction should be correct and based on valid evidence. Yet, for VLMs, the impact of fine-tuning on prediction rationality is seldomly investigated. To study this problem, we proposed two new metrics called Prediction Trustworthiness and Inference Reliability. We conducted extensive experiments on various settings and observed some interesting phenomena. On the one hand, we found that the well-adopted fine-tuning methods led to more correct predictions based on invalid evidence. This potentially undermines the trustworthiness of correct predictions from fine-tuned VLMs. On the other hand, having identified valid evidence of target objects, fine-tuned VLMs were more likely to make correct predictions. Moreover, the findings are also consistent under distributional shifts and across various experimental settings. We hope our research offer fresh insights to VLM fine-tuning.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。