用外部知识减少视觉问答中的幻觉，提升答案准确性。
FilterRAG: Zero-Shot Informed Retrieval-Augmented Generation to Mitigate Hallucinations in VQA
- 结合视觉与文本模型，通过检索外部知识约束回答生成。
- 在OK-VQA上准确率达36.5%，显著降低幻觉率。
- 适合需高可信度的现实场景视觉问答系统。
视觉问答要求模型整合视觉与文本理解生成准确答案，但现有模型仍易产生幻觉，尤其在知识依赖或分布外场景中。本文提出FilterRAG，将BLIP-VQA与检索增强生成结合，利用Wikipedia和DBpedia等外部知识源对答案进行约束。该方法在OK-VQA数据集上达到36.5%的准确率，证明其在领域内与分布外设置下均能有效减少幻觉、提升鲁棒性，具备实际部署潜力。
原文摘要 · Abstract (English)
Visual Question Answering requires models to generate accurate answers by integrating visual and textual understanding. However, VQA models still struggle with hallucinations, producing convincing but incorrect answers, particularly in knowledge-driven and Out-of-Distribution scenarios. We introduce FilterRAG, a retrieval-augmented framework that combines BLIP-VQA with Retrieval-Augmented Generation to ground answers in external knowledge sources like Wikipedia and DBpedia. FilterRAG achieves 36.5% accuracy on the OK-VQA dataset, demonstrating its effectiveness in reducing hallucinations and improving robustness in both in-domain and Out-of-Distribution settings. These findings highlight the potential of FilterRAG to improve Visual Question Answering systems for real-world deployment.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。