提出可解释的视觉语言模型框架，有效减少幻觉并提升推理可信度。
CaVe-VLM-CoT: An Interpretable Vision-Language Model Framework
- 五阶段闭环流程强制每步推理有证据支持，错误自动触发重检索
- 在ScienceQA上达87.1%准确率，CaVeScore达56.6%，优于现有方法
- 专设23项指标评估检索、引用与跨模态对齐，适合可信AI研究者
视觉语言模型仍易产生幻觉，生成流畅但视觉不符的输出。现有链式思维与检索增强方法仅部分缓解问题，因未实现步骤级引用定位，也无法将验证失败反馈至检索环节进行修正。本文提出CaVe-VLM-CoT，一个基于反思的模块化代理-检索增强框架，通过五阶段闭环流程（提取器、检索器、求解器、引用注入器、验证器）实现证据驱动推理：检测到无依据陈述时，会结构化反馈至提取器，触发针对性重检索。由于现有框架未能联合衡量检索质量、步骤引用忠实度与跨模态对齐，我们设计一套涵盖各阶段的23项组件级指标，核心为CaVeScore——综合考虑准确率、引用精确率与召回率、归属性及证据对齐性。无需架构或提示修改，CaVe-VLM-CoT在ScienceQA上实现87.1%准确率和56.6% CaVeScore；在包含30个学科的MMMU上达成55.2%准确率和35.7% CaVeScore。
原文摘要 · Abstract (English)
Vision-Language Models (VLMs) remain prone to hallucinations, producing fluent but visually unfaithful outputs. Existing chain-of-thought and retrieval-augmented methods only partially address this, as they neither enforce step-level citation grounding nor route verification failures back to retrieval for correction. We present CaVe-VLM-CoT, a modular reflection-based agentic-RAG framework that enforces evidence-grounded reasoning through a five-stage closed-loop pipeline: Extractor, Retriever, Solver, Citation Injector, and Verifier, in which detected ungrounded claims trigger structured feedback to the Extractor for targeted re-retrieval. Since no existing framework jointly measures retrieval quality, step-wise citation faithfulness, and cross-modal grounding, we propose a suite of 23 component-wise metrics across all stages, anchored by CaVeScore, a composite metric weighting accuracy, citation precision and recall, attribution, and evidence grounding. Without any architectural or prompt modifications, CaVe-VLM-CoT achieves 87.1\% accuracy and 56.6\% CaVeScore on ScienceQA , and 55.2\% accuracy and 35.7\% CaVeScore on MMMU (30 subjects).
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。