融合检索与生成，提升伊斯兰知识问答准确率
Transformer Tafsir at QIAS 2025 Shared Task: Hybrid Retrieval-Augmented Generation for Islamic Knowledge Question Answering
- 三阶段检索增强生成：关键词+语义匹配+精排重排序
- 最高提升25%准确率，子任务二达80%正确率
- 适合需要精准宗教知识推理的AI系统开发者
本文介绍了我们在QIAS 2025共享任务中针对伊斯兰知识理解与推理的提交方案。我们构建了一个混合检索增强生成（RAG）系统，结合稀疏与稠密检索方法，并采用交叉编码器重排序以提升大语言模型（LLM）性能。该三阶段流程包含BM25初始检索、稠密嵌入模型语义匹配以及交叉编码器精排。我们在两个子任务上使用Fanar和Mistral两套LLM进行评估，结果表明该RAG流程显著提升性能，准确率最高提升达25%，具体表现取决于任务与模型配置。最优配置为Fanar模型，在子任务一中达到45%准确率，子任务二达80%。
原文摘要 · Abstract (English)
This paper presents our submission to the QIAS 2025 shared task on Islamic knowledge understanding and reasoning. We developed a hybrid retrieval-augmented generation (RAG) system that combines sparse and dense retrieval methods with cross-encoder reranking to improve large language model (LLM) performance. Our three-stage pipeline incorporates BM25 for initial retrieval, a dense embedding retrieval model for semantic matching, and cross-encoder reranking for precise content retrieval. We evaluate our approach on both subtasks using two LLMs, Fanar and Mistral, demonstrating that the proposed RAG pipeline enhances performance across both, with accuracy improvements up to 25%, depending on the task and model configuration. Our best configuration is achieved with Fanar, yielding accuracy scores of 45% in Subtask 1 and 80% in Subtask 2.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。