用大模型提炼逻辑规则，让视觉问答更可解释且易扩展
Distilling Answer-Set Programming Rules from LLMs for Neurosymbolic Visual Question Answering
- 通过提示大模型扩展答案集程序来生成推理规则
- 仅需少量示例即可让大模型生成正确规则
- 适合需要可解释推理的视觉问答场景
视觉问答（VQA）要求整合多模态输入并进行推理。基于逻辑的模块化方法在可解释性上优于端到端训练系统，但任务变更时规则调整成本高。本文提出从大语言模型（LLMs）中蒸馏规则的方法：通过提示LLM扩展初始的VQA推理理论（以答案集程序表达），利用VQA数据集中的示例引导、验证结果，并借助ASP求解器反馈纠正错误规则。实验表明该方法在多个VQA数据集上有效，且仅需少量示例即可获得正确规则。结果表明，从大模型中蒸馏规则是传统数据驱动规则学习的有力替代方案。
原文摘要 · Abstract (English)
Visual Question Answering (VQA) is the task of answering questions about images, requiring the integration of multimodal input and reasoning. Modular approaches that incorporate logic-based representations into the reasoning component offer clear advantages over end-to-end trained systems, particularly in terms of interpretability. However, adapting or extending these representations when task requirements change can place a significant burden on developers. To address this challenge, we present an approach for distilling rules from Large Language Models (LLMs). Our method prompts an LLM to extend an initial VQA reasoning theory, expressed as an answer-set program, to meet new requirements of the task. Examples from VQA datasets guide the LLM, validate the results, and help correct erroneous rules by leveraging feedback from the ASP solver. We demonstrate that our approach is effective across diverse VQA datasets. Notably, only a few examples are needed to elicit correct rules from LLMs. Our experiments suggest that rule distillation from LLMs is a promising alternative to traditional data-driven rule learning approaches. Under consideration in Theory and Practice of Logic Programming (TPLP).
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。