用逻辑得分融合检索结果，提升开放域问答准确率
LoRE: Logit-Ranked Retriever Ensemble for Enhancing Open-Domain Question Answering
- 集成多种检索器，用大模型逻辑得分重排结果
- 在SQuAD上准确率提升22.83%，F1提升14.95%
- 适合处理复杂查询的高精度问答系统
基于检索的问答系统常受位置偏差影响，导致答案生成不优。本文提出LoRE（Logit-Ranked Retriever Ensemble），通过整合多样化的检索器（如BM25和带FAISS索引的句子嵌入模型），并引入基于逻辑得分的答案重排算法，有效缓解位置偏差。该算法结合大语言模型的逻辑得分与文档检索排名，显著提升答案相关性与准确性。在NarrativeQA和SQuAD数据集上的实验表明，LoRE在精确匹配（EM）和F1分数上均显著优于现有方法。在SQuAD上，其ROUGE-L、EM和F1分别较基线提升14.5%、22.83%和14.95%。定性分析显示，对复杂查询生成的答案更具相关性和准确性。
原文摘要 · Abstract (English)
Retrieval-based question answering systems often suffer from positional bias, leading to suboptimal answer generation. We propose LoRE (Logit-Ranked Retriever Ensemble), a novel approach that improves answer accuracy and relevance by mitigating positional bias. LoRE employs an ensemble of diverse retrievers, such as BM25 and sentence transformers with FAISS indexing. A key innovation is a logit-based answer ranking algorithm that combines the logit scores from a large language model (LLM), with the retrieval ranks of the passages. Experimental results on NarrativeQA, SQuAD demonstrate that LoRE significantly outperforms existing retrieval-based methods in terms of exact match and F1 scores. On SQuAD, LoRE achieves 14.5\%, 22.83\%, and 14.95\% improvements over the baselines for ROUGE-L, EM, and F1, respectively. Qualitatively, LoRE generates more relevant and accurate answers, especially for complex queries.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。