让视觉问答模型像人一样分步推理，还能看懂每一步的依据。
Visually Interpretable Subtask Reasoning for Visual Question Answering
- 用分步思维链替代黑箱推理，结合文本与视觉解释。
- 在两个数据集上准确率提升，且推理过程可读性强。
- 适合需要透明决策过程的AI应用，如医疗或教育。
回答如“哪些红色家具可以坐？”这类复杂视觉问题，需经历物体识别、属性筛选和关系理解等多步推理。现有方法通过将任务分解为子任务程序提升多模态大模型（MLLMs）的可解释性，但存在计算开销大、对目标数据适应性差等问题。为此，我们提出VISTAR（Visually Interpretable Subtask-Aware Reasoning Model），一种以子任务驱动的训练框架，通过在MLLM中生成文本与视觉解释，同时提升可解释性与推理能力。VISTAR不依赖外部模型，而是微调MLLM以生成结构化的“思维子任务”（Subtask-of-Thought）推理序列。在两个基准上的实验表明，VISTAR在保持可解释性的前提下持续提升推理准确率。代码与数据集将开源于https://github.com/ChengJade/VISTAR。
原文摘要 · Abstract (English)
Answering complex visual questions like `Which red furniture can be used for sitting?' requires multi-step reasoning, including object recognition, attribute filtering, and relational understanding. Recent work improves interpretability in multimodal large language models (MLLMs) by decomposing tasks into sub-task programs, but these methods are computationally expensive and less accurate due to poor adaptation to target data. To address this, we introduce VISTAR (Visually Interpretable Subtask-Aware Reasoning Model), a subtask-driven training framework that enhances both interpretability and reasoning by generating textual and visual explanations within MLLMs. Instead of relying on external models, VISTAR fine-tunes MLLMs to produce structured Subtask-of-Thought rationales (step-by-step reasoning sequences). Experiments on two benchmarks show that VISTAR consistently improves reasoning accuracy while maintaining interpretability. Our code and dataset will be available at https://github.com/ChengJade/VISTAR.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。