通过多嵌入与置信度评分，提升大模型解数学题的准确率。
Confident RAG: Enhancing the Performance of LLMs for Mathematics Question Answering through Multi-Embedding and Confidence Scoring
- 用多个嵌入模型生成答案，选置信度最高的
- 相比基础大模型提升约10%准确率
- 适合需要可靠数学推理的教育AI系统
大语言模型在数学教育中潜力巨大，但复杂推理能力不足。检索增强生成（RAG）虽能引入外部知识，但性能依赖单一嵌入模型，效果不稳定。本文借鉴智能体工作流思想，提出两种结合多嵌入模型的方法。混合嵌入RAG（融合检索文档）提升有限；而置信度RAG（生成多答案并选最高置信度）表现显著。实验显示，置信度RAG相较基础LLM平均准确率提升约10%，相较基础RAG提升5%。该方法在不同LLM和嵌入模型下均保持稳定，是一种即插即用的可靠数学推理方案。研究为教育场景中部署自主规划、迭代优化的智能体式RAG系统奠定了基础。
原文摘要 · Abstract (English)
Large Language Models (LLMs) hold significant promise for mathematics education, yet they often struggle with complex mathematical reasoning. While Retrieval-Augmented Generation (RAG) mitigates these issues by grounding LLMs in external knowledge, its effectiveness remains unstable, heavily dependent on the choice of a single embedding model. Moving beyond static RAG workflows, we draw on agentic workflow patterns, a paradigm that introduces structured task decomposition and collaboration to enhance system performance. We propose and examine two novel approaches that combine the benefits of multiple embedding models. While our Mixture-Embedding RAG approach (fusing retrieved documents) shows limited gains, our Confident RAG method (generating multiple answers and selecting the one with the highest confidence score) demonstrates significant improvement. Experimental results show that Confident RAG achieved average accuracy improvements of approximately 10% over vanilla LLMs and 5% over vanilla RAG. The consistent results across different LLMs and embedding models indicate that Confident RAG is an efficient plug-and-play solution for trustworthy mathematical AI assistants. Finally, we discuss how this work lays the groundwork for deploying Agentic RAG systems in educational settings, where autonomous planning and iterative refinement can be built upon our robust retrieval foundation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。