让大模型自己教自己推理,提升视觉问答准确率
From Hindsight to Foresight: Self-Encouraged Hindsight Distillation for Knowledge-based Visual Question Answering
- 用事后推理生成教师信号,引导模型学会分步思考
- 7-8B模型不依赖外部知识库,性能超越现有方法
- 适合想提升模型可解释性与推理能力的研究者
基于知识的视觉问答(KBVQA)需要在跨模态理解之外引入外部知识。现有方法或通过上下文学习利用多模态大模型中的隐式知识,或通过检索增强生成使用显式知识,但其推理过程仍不透明、缺乏明确的多步轨迹。为此,我们提出自激励的事后蒸馏推理框架(HinD),旨在通过构建带有特权信息的“事后教师”来激发多模态大模型内部的推理能力。首先,以推理目标为特权信息提示大模型完成推理,生成无需答案的“事后零数据”。随后,“前瞻学生”通过两种方式从“事后教师”中学习:(1)事后蒸馏微调,将“事后零数据”自蒸馏为模块化的思维链生成器与知识生成器,分别用于顺序步骤和离散事实生成;(2)知识鼓励偏好优化,鼓励大模型中低置信度但相关的知识,抑制高置信度但无关的知识。在OK-VQA和A-OKVQA上的实验验证了该方法的有效性,表明仅使用7-8B规模的多模态大模型即可实现优越性能,无需商业模型接口或外部知识检索。
原文摘要 · Abstract (English)
Knowledge-based Visual Question Answering (KBVQA) necessitates external knowledge incorporation beyond cross-modal understanding. Existing KBVQA methods either utilize implicit knowledge in multimodal large language models via in-context learning or explicit knowledge via retrieval augmented generation. However, their reasoning processes remain implicit, without explicit multi-step trajectories. To address this gap, we propose a Self-Encouraged Hindsight Distillation Reasoning (HinD) framework, aiming at eliciting reasoning ability inside the MLLM by constructing a Hindsight Teacher with privileged information to teach the Foresight Student. First, we construct the Hindsight Teacher by prompting the MLLM with the reasoning target as privileged information to complete the reasoning process, obtaining Hindsight-Zero training data. Then, the Foresight Student, without knowing the answer, learns the golden trajectories from Hindsight in two ways: (1) Hindsight Distillation Fine-Tuning to self-distill the Hindsight-Zero into a modularized Chain-of-Thought Generator and a Knowledge Generator for sequential steps and discrete facts generation, respectively; (2) Knowledge Encouragement Preference Optimization to encourage the under-confident but relevant knowledge inside the MLLM and suppress the over-confident but irrelevant one. Experiments on OK-VQA and A-OKVQA validate the effectiveness of HinD, showing that HinD with 7-8B MLLM achieves superior performance without commercial model APIs or retrieved knowledge.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。