让重写器先读专业文档，提升跨领域问答准确率
Read the Docs Before Rewriting: Equip Rewriter with Domain Knowledge via Continual Pre-training
- 重写前持续预训练专业文档，模拟备考策略
- 多领域测试中显著缩小查询与文档的语义差距
- 适合需要精准专业知识的行业问答场景
基于检索增强生成（RAG）的问答系统通过检索相关文档来增强大模型的知识。当用户查询与文档表述不一致时，常需对查询进行重写。然而在专业领域，重写模型因缺乏领域知识而表现受限。为此，我们提出R&R（Read the doc before Rewriting）重写器，通过在专业文档上进行持续预训练，类似学生备考开放书考试前复习教材。该方法还可结合监督微调进一步提升性能。在多个数据集上的实验表明，R&R在多领域专业问答中表现优异，有效弥合查询与文档间的语义鸿沟，同时保持通用场景下的良好性能，推动了RAG问答系统在专业领域的应用。
原文摘要 · Abstract (English)
A Retrieval-Augmented Generation (RAG)-based question-answering (QA) system enhances a large language model's knowledge by retrieving relevant documents based on user queries. Discrepancies between user queries and document phrasings often necessitate query rewriting. However, in specialized domains, the rewriter model may struggle due to limited domain-specific knowledge. To resolve this, we propose the R\&R (Read the doc before Rewriting) rewriter, which involves continual pre-training on professional documents, akin to how students prepare for open-book exams by reviewing textbooks. Additionally, it can be combined with supervised fine-tuning for improved results. Experiments on multiple datasets demonstrate that R\&R excels in professional QA across multiple domains, effectively bridging the query-document gap, while maintaining good performance in general scenarios, thus advancing the application of RAG-based QA systems in specialized fields.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。