让大模型学会说‘不知道’，提升回答可信度
Delineating Knowledge Boundaries for Honest Large Vision-Language Models
- 构建专属数据集，通过多样本一致性判断知识边界
- 微调后拒绝率提升，真实准确率从57.9%增至67.3%
- 适用于医疗等专业领域，适合追求可信AI的开发者
大型视觉语言模型虽在多模态任务中表现优异，却易产生事实幻觉，尤其在长尾或专业领域。当前模型缺乏拒绝超出参数知识范围问题的能力。本文提出系统性框架，增强模型对未知问题的拒绝能力。首先，利用多样本一致性探测构建模型专属的「Visual-Idk」数据集，区分已知与未知事实；随后通过监督微调结合偏好优化（如DPO、ORPO）对模型进行对齐，明确其知识边界。在Visual-Idk数据集上的实验显示，该方法将真实准确率从57.9%提升至67.3%。内部探测结果表明，模型真正识别自身边界，而非仅记忆拒绝模式。该框架还成功泛化至分布外的医学与感知领域，为更可信、审慎的视觉助手提供可靠路径。
原文摘要 · Abstract (English)
Large Vision-Language Models (VLMs) have achieved remarkable multimodal performance yet remain prone to factual hallucinations, particularly in long-tail or specialized domains. Moreover, current models exhibit a weak capacity to refuse queries that exceed their parametric knowledge. In this paper, we propose a systematic framework to enhance the refusal capability of VLMs when facing such unknown questions. We first curate a model-specific "Visual-Idk" (Visual-I don't know) dataset, leveraging multi-sample consistency probing to distinguish between known and unknown facts. We then align the model using supervised fine-tuning followed by preference-aware optimization (e.g., DPO, ORPO) to effectively delineate its knowledge boundaries. Results on the Visual-Idk dataset show our method improves the Truthful Rate from 57.9\% to 67.3\%. Additionally, internal probing also demonstrates that the model genuinely recognizes its boundaries instead of just memorizing refusal patterns. Our framework further generalizes to out-of-distribution medical and perceptual domains, providing a robust path toward more trustworthy and prudent visual assistants.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。