针对视觉问答幻觉问题，提出三步综合框架提升多模态理解能力。
Solution for Meta KDD Cup'25: A Comprehensive Three-Step Framework for Vision Question Answering
- 基于RAG与重排序技术增强视觉上下文理解
- 单模型完成三项任务，第三项获自动评估第一
- 适合关注多模态生成与检索的开发者
视觉大语言模型（VLLMs）虽提升了多模态理解与视觉问答（VQA）能力，但仍存在生成幻觉问题。多模态检索增强生成（RAG）通过引入外部信息缓解该问题，但在视觉上下文理解、多源检索和多轮交互方面仍面临挑战。为此，Meta构建了CRAG-MM基准并举办KDD Cup 2025 CRAG-MM挑战赛，包含三项任务。本文介绍BlackPearl团队在该赛事中所有任务的解决方案。我们为每项任务采用单一模型，核心方法包括数据增强、RAG、重排序及多任务微调。方案在三项任务的自动评估中分别获得第3、第3、第1名，任务3在人工评估中位列第二。
原文摘要 · Abstract (English)
Vision Large Language Models (VLLMs) have improved multi-modal understanding and visual question answering (VQA), but still suffer from hallucinated answers. Multi-modal Retrieval-Augmented Generation (RAG) helps address these issues by incorporating external information, yet challenges remain in visual context comprehension, multi-source retrieval, and multi-turn interactions. To address these challenges, Meta constructed the CRAG-MM benchmark and launched the CRAG-MM Challenge at KDD Cup 2025, which consists of three tasks. This paper describes the solutions of all tasks in Meta KDD Cup'25 from BlackPearl team. We use a single model for each task, with key methods including data augmentation, RAG, reranking, and multi-task fine-tuning. Our solution achieve automatic evaluation rankings of 3rd, 3rd, and 1st on the three tasks, and win second place in Task3 after human evaluation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。