评测顶尖RAG系统在真实问答任务中的表现，发现结合BGE重排序的Pinecone检索最优。
RAGtifier: Evaluating RAG Generation Approaches of State-of-the-Art RAG Systems for the SIGIR LiveRAG Competition
- 采用InstructRAG框架，用Pinecone稠密索引+ BGE重排序提升检索精度。
- 在DataMorgana数据集上取得1.13正确率和0.55忠实度，排名第三。
- 适用于需要高准确率、低幻觉的开放域问答系统开发与评估。
检索增强生成（RAG）通过融合大语言模型的参数化知识与外部非参数化信息源，旨在提升事实准确性并减少幻觉。2025年SIGIR LiveRAG挑战赛聚焦于在DataMorgana QA数据集上最大化准确率，该数据集包含单跳与多跳问题。参赛者可使用Fineweb 10BT数据集的稀疏OpenSearch与稠密Pinecone索引，模型限制为不超过100亿参数，最终答案生成采用Falcon-3-10B。评估由判别型LLM与人工评价员共同完成。通过探索不同检索器组合与RAG方案，最终方案采用InstructRAG结合Pinecone检索器与BGE重排序器，在非人类评估中取得1.13的正确率与0.55的忠实度，总排名第三。
原文摘要 · Abstract (English)
Retrieval-Augmented Generation (RAG) enriches Large Language Models (LLMs) by combining their internal, parametric knowledge with external, non-parametric sources, with the goal of improving factual correctness and minimizing hallucinations. The LiveRAG 2025 challenge explores RAG solutions to maximize accuracy on DataMorgana's QA pairs, which are composed of single-hop and multi-hop questions. The challenge provides access to sparse OpenSearch and dense Pinecone indices of the Fineweb 10BT dataset. It restricts model use to LLMs with up to 10B parameters and final answer generation with Falcon-3-10B. A judge-LLM assesses the submitted answers along with human evaluators. By exploring distinct retriever combinations and RAG solutions under the challenge conditions, our final solution emerged using InstructRAG in combination with a Pinecone retriever and a BGE reranker. Our solution achieved a correctness score of 1.13 and a faithfulness score of 0.55 in the non-human evaluation, placing it overall in third place in the SIGIR 2025 LiveRAG Challenge.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。