提出问答聚焦的跨文档知识筛选方法，提升视觉问答准确率。
QKVQA: Question-Focused Filtering for Knowledge-based VQA
- 设计可训练的问答聚焦过滤器，关注问题相关图文内容
- 跨文章动态选择知识片段，准确率分别提升3.2%和2.2%
- 适用于需外部知识的复杂视觉问答任务
视觉问答（VQA）是基于图像内容回答问题的任务。在此基础上，基于知识的视觉问答（KB-VQA）要求模型回答依赖于图像之外的外部知识的问题。此类场景下，有效的知识过滤对高准确率至关重要。传统方法存在两大缺陷：在候选段落编码时未能聚焦与问题相关的部分，且仅用相似度指标从单篇文章中定位段落，导致信息受限。为此，本文提出一种面向问题的跨文章知识过滤方法。具体包括可训练的问答聚焦过滤器（QFF）和基于块的动态跨文章选择模块（CDA）。该方法在保持推理时间接近最优方法的同时，显著缩短上下文长度，高效获取高质量过滤知识。在Encyclopedic-VQA和InfoSeek数据集上，准确率分别超过当前最佳方法3.2和2.2个百分点。代码已公开：https://github.com/leaffeall/QKVQA。
原文摘要 · Abstract (English)
Visual Question Answering (VQA) is the task of answering questions based on image content. Building upon this, Knowledge-Based VQA (KB-VQA) requires models to answer questions that depend on external knowledge beyond the visual content of an image. In such settings, effective knowledge filtering is essential for achieving high question answering accuracy. Typical filtering methods suffer from two issues: they fail to focus on parts relevant to the question during candidate section encoding, and they use similarity metrics to locate a section from a single article, resulting in information limitation. To address these issues, this paper proposes a question-focused, cross-article filtering method. Specifically, we design a trainable Question-Focused Filter (QFF) and a Chunk-based Dynamic Cross-Article Selection module (CDA). This approach maintains inference time comparable to the optimal method with the shorter context length, efficiently obtaining high-quality filtered knowledge. The accuracy outperforms current state-of-the-art methods by 3.2 and 2.2 percentage points on Encyclopedic-VQA and InfoSeek, respectively. The code is publicly available at: https://github.com/leaffeall/QKVQA.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。