通过动态写笔记提升大模型在复杂问答中的推理能力
Augmenting LLM Reasoning with Dynamic Notes Writing for Complex QA
- 每轮检索后生成简洁摘要笔记，过滤无关信息
- 在4个数据集上平均提升15.6个百分点，输出量几乎不变
- 无需微调，适配多种迭代RAG框架，适合长文本推理任务
多跳问答中的迭代RAG面临上下文过长和冗余信息累积的问题，影响模型处理与推理能力。现有方法或局限于单轮RAG、需微调，或在迭代RAG中缺乏可扩展性。为此，我们提出Notes Writing：在每一步从检索文档中生成简洁相关笔记，减少噪声并保留关键信息。该方法间接扩展了大语言模型的有效上下文长度，使其能更有效地进行推理与规划，同时处理更大输入量。Notes Writing具有框架无关性，可集成于多种迭代RAG方法中。我们在三种迭代RAG方法、两个模型及四个评估数据集上验证其有效性，结果表明平均提升15.6个百分点，输出令牌数增加极少。
原文摘要 · Abstract (English)
Iterative RAG for multi-hop question answering faces challenges with lengthy contexts and the buildup of irrelevant information. This hinders a model's capacity to process and reason over retrieved content and limits performance. While recent methods focus on compressing retrieved information, they are either restricted to single-round RAG, require finetuning or lack scalability in iterative RAG. To address these challenges, we propose Notes Writing, a method that generates concise and relevant notes from retrieved documents at each step, thereby reducing noise and retaining only essential information. This indirectly increases the effective context length of Large Language Models (LLMs), enabling them to reason and plan more effectively while processing larger volumes of input text. Notes Writing is framework agnostic and can be integrated with different iterative RAG methods. We demonstrate its effectiveness with three iterative RAG methods, across two models and four evaluation datasets. Notes writing yields an average improvement of 15.6 percentage points overall, with minimal increase in output tokens.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。