用大模型自动提炼视觉问答的可解释规则，提升系统透明度。
Declarative Knowledge Distillation from Large Language Models for Visual Question Answering Datasets
- 通过提示大模型扩展初始逻辑规则，生成可解释的推理程序。
- 在CLEVR和GQA数据集上验证，规则准确率显著优于传统方法。
- 适合需要透明决策过程的AI应用，如医疗或金融问答系统。
视觉问答（VQA）需对图像和问题进行多模态处理与推理以得出答案。采用声明式表示的模块化系统相比端到端训练系统更具可解释性优势，但手工设计规则成本高。本文提出一种从大型语言模型（LLMs）中进行声明式知识蒸馏的方法：给定一个初始的答案集程序（ASP），通过VQA数据集中的示例引导大模型扩展该理论，以满足任务需求。利用ASP求解器反馈验证并修正规则，确保其正确性。实验表明，该方法在主流的CLEVR和GQA数据集上均有效，证明了从大模型蒸馏知识是继数据驱动规则学习之外的一条有前景路径。
原文摘要 · Abstract (English)
Visual Question Answering (VQA) is the task of answering a question about an image and requires processing multimodal input and reasoning to obtain the answer. Modular solutions that use declarative representations within the reasoning component have a clear advantage over end-to-end trained systems regarding interpretability. The downside is that crafting the rules for such a component can be an additional burden on the developer. We address this challenge by presenting an approach for declarative knowledge distillation from Large Language Models (LLMs). Our method is to prompt an LLM to extend an initial theory on VQA reasoning, given as an answer-set program, to meet the requirements of the VQA task. Examples from the VQA dataset are used to guide the LLM, validate the results, and mend rules if they are not correct by using feedback from the ASP solver. We demonstrate that our approach works on the prominent CLEVR and GQA datasets. Our results confirm that distilling knowledge from LLMs is in fact a promising direction besides data-driven rule learning approaches.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。