arXiv:2509.09159cs.CVcs.AI2025-09中稿 · the IEEE Internati…

解决视觉问答中的知识噪声问题,提升答案准确性

A Knowledge Noise Mitigation Framework for Knowledge-based Visual Question Answering

  • 通过图像-问题对提炼关键信息,生成低噪查询以提高知识相关性
  • 利用大模型筛选出对答案有益的知识片段,减少冗余信息干扰
  • 仅在模型不确定时才引入知识,有效降低噪声影响,适合高精度场景

基于知识的视觉问答(KB-VQA)要求模型理解图像并利用外部知识提供准确答案。现有方法常直接将检索到的知识注入模型,忽视了知识冗余带来的噪声。为此,本文提出一种无需训练的框架,通过知识聚焦机制缓解噪声影响:首先,从图像-问题对中提取关键内容,生成低噪声查询,提升相关知识的检索效果;其次,利用大模型识别并提取知识中有助于回答的片段;最后,设计选择性知识融合策略,仅在模型对答案不确定时引入知识,从而减轻冗余信息的影响。实验表明,该框架显著优于现有最先进方法。

原文摘要 · Abstract (English)

Knowledge-based visual question answering (KB-VQA) requires a model to understand images and utilize external knowledge to provide accurate answers. Existing approaches often directly augment models with retrieved information from knowledge sources while ignoring substantial knowledge redundancy, which introduces noise into the answering process. To address this, we propose a training-free framework with knowledge focusing for KB-VQA, that mitigates the impact of noise by enhancing knowledge relevance and reducing redundancy. First, for knowledge retrieval, our framework concludes essential parts from the image-question pairs, creating low-noise queries that enhance the retrieval of highly relevant knowledge. Considering that redundancy still persists in the retrieved knowledge, we then prompt large models to identify and extract answer-beneficial segments from knowledge. In addition, we introduce a selective knowledge integration strategy, allowing the model to incorporate knowledge only when it lacks confidence in answering the question, thereby mitigating the influence of redundant information. Our framework enables the acquisition of accurate and critical knowledge, and extensive experiments demonstrate that it outperforms state-of-the-art methods.

视觉问答知识增强噪声抑制

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。