用课程学习优化强化学习，提升多模态问答准确率。
A Curriculum Learning Approach to Reinforcement Learning: Leveraging RAG for Multimodal Question Answering
- 结合课程学习与强化学习，逐步提升模型答题能力
- 任务1准确率领先第二名52.38%，显著降低幻觉
- 适合需要多轮对话和跨源信息融合的场景
本文介绍点评安全团队在META CRAG-MM挑战赛中的解决方案。该挑战要求构建一个完整的检索增强生成系统，支持多模态多轮问答。比赛包含三个任务：（1）基于图像构建的知识图谱中检索结构化数据回答问题；（2）融合知识图谱与网络搜索结果进行信息合成；（3）处理需上下文理解与多源信息聚合的多轮对话。针对任务1，方案基于视觉大语言模型，通过GPT-4.1知识蒸馏进行监督微调，并引入课程学习策略引导强化学习，显著提升答案准确率并减少幻觉。任务2和任务3进一步接入网络搜索API，增强外部知识获取能力。最终方案在任务1中以52.38%的显著优势获得第一名，在任务3中位列第三，验证了课程学习与强化学习融合的有效性。
原文摘要 · Abstract (English)
This paper describes the solutions of the Dianping-Trust-Safety team for the META CRAG-MM challenge. The challenge requires building a comprehensive retrieval-augmented generation system capable for multi-modal multi-turn question answering. The competition consists of three tasks: (1) answering questions using structured data retrieved from an image-based mock knowledge graph, (2) synthesizing information from both knowledge graphs and web search results, and (3) handling multi-turn conversations that require context understanding and information aggregation from multiple sources. For Task 1, our solution is based on the vision large language model, enhanced by supervised fine-tuning with knowledge distilled from GPT-4.1. We further applied curriculum learning strategies to guide reinforcement learning, resulting in improved answer accuracy and reduced hallucination. For Task 2 and Task 3, we additionally leveraged web search APIs to incorporate external knowledge, enabling the system to better handle complex queries and multi-turn conversations. Our approach achieved 1st place in Task 1 with a significant lead of 52.38%, and 3rd place in Task 3, demonstrating the effectiveness of the integration of curriculum learning with reinforcement learning in our training pipeline.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。