让AI自动改写难答问题,提升问答准确率。
DRS: Deep Question Reformulation With Structured Output
- 用深度搜索算法结合实体约束,结构化生成新问题
- GPT-3.5改写准确率从23%升至70.42%
- 适合需要精准提问的文档问答场景
问答是大语言模型的核心能力。但当用户对文本内容理解不足时,常提出无法回答的问题。尽管现有研究发现大模型能识别无解问题,却难以有效协助用户重述问题。即使先进模型如GPT-3.5,在此任务上也表现有限。为此,我们提出DRS:基于结构化输出的深度问题重述方法,旨在提升大模型在新文档中辅助用户重述问题的能力。DRS融合大模型与基于深度优先搜索的算法,迭代探索潜在实体组合,并通过预定义实体约束输出,实现结构化重述。实验表明,DRS将GPT-3.5的问题重述准确率从23.03%提升至70.42%,同时显著改善开源模型Gemma2-9B的表现,从26.35%提升至56.75%。
原文摘要 · Abstract (English)
Question answering represents a core capability of large language models (LLMs). However, when individuals encounter unfamiliar knowledge in texts, they often formulate questions that the text itself cannot answer due to insufficient understanding of the underlying information. Recent studies reveal that while LLMs can detect unanswerable questions, they struggle to assist users in reformulating these questions. Even advanced models like GPT-3.5 demonstrate limited effectiveness in this regard. To address this limitation, we propose DRS: Deep Question Reformulation with Structured Output, a novel zero-shot method aimed at enhancing LLMs ability to assist users in reformulating questions to extract relevant information from new documents. DRS combines the strengths of LLMs with a DFS-based algorithm to iteratively explore potential entity combinations and constrain outputs using predefined entities. This structured approach significantly enhances the reformulation capabilities of LLMs. Comprehensive experimental evaluations demonstrate that DRS improves the reformulation accuracy of GPT-3.5 from $23.03\%$ to $70.42\%$, while also enhancing the performance of open-source models, such as Gemma2-9B, from $26.35\%$ to $56.75\%$.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。