用推理框架+解码策略，让大模型回答更靠谱。
An Analysis of Decoding Methods for LLM-based Agents for Faithful Multi-Hop Question Answering
- 结合ReAct框架与忠实性解码法提升答案可信度
- 在HotpotQA上F1从19.5升至32.6
- 适合关注大模型幻觉问题的研究者
大语言模型常产生事实错误，即幻觉，限制其在知识密集型自然语言任务中的准确性。检索增强生成和代理框架（如ReAct）通过引入外部知识缓解此问题，但模型仍易偏离检索内容。近期研究探索了无需训练的解码策略以提升忠实性。本文系统分析了ReAct框架与DeCoRe、DoLa、CAD等解码策略组合对多跳问答任务中生成答案忠实性的影响。结果表明，将代理式知识检索与增强忠实性的解码方法结合，可显著提升下游任务准确率。例如，在HotpotQA数据集上，使用ReAct与DoLa使F1值从19.5提升至32.6。
原文摘要 · Abstract (English)
Large Language Models (LLMs) frequently produce factually inaccurate outputs - a phenomenon known as hallucination - which limits their accuracy in knowledge-intensive NLP tasks. Retrieval-augmented generation and agentic frameworks such as Reasoning and Acting (ReAct) can address this issue by giving the model access to external knowledge. However, LLMs often fail to remain faithful to retrieved information. Mitigating this is critical, especially if LLMs are required to reason about the retrieved information. Recent research has explored training-free decoding strategies to improve the faithfulness of model generations. We present a systematic analysis of how the combination of the ReAct framework and decoding strategies (i.e., DeCoRe, DoLa, and CAD) can influence the faithfulness of LLM-generated answers. Our results show that combining an agentic framework for knowledge retrieval with decoding methods that enhance faithfulness can increase accuracy on the downstream Multi-Hop Question Answering tasks. For example, we observe an F1 increase from 19.5 to 32.6 on HotpotQA when using ReAct and DoLa.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。