用图推理提升大模型长文本理解能力,减少幻觉。
Joint Enhancement of Relational Reasoning for Long-Context LLMs
- 构建动态图结构,分块摘要+有向无环图去重
- 引入蒙特卡洛树搜索,提升复杂推理准确率
- 适合需要高透明度和长文理解的场景
尽管取得显著进展,大语言模型在处理长文本时仍受限于记忆瓶颈,难以应对复杂任务。同时,模型缺乏透明性且易产生幻觉。为此,我们提出JERR框架,通过图式推理增强长文本理解能力。该框架包含三部分:分块提取摘要,构建有向无环图(DAG)消除冗余以保证逻辑一致性,以及引入蒙特卡洛树搜索(MCTS)辅助模型探索复杂推理路径。实验表明,JERR在ROUGE和F1指标上持续优于所有基线,在LLM-Rater评估中取得最高分,显著提升了模型的可靠性与可解释性。
原文摘要 · Abstract (English)
Despite significant progress, large language models (LLMs) still struggle with long contexts due to memory limitations and their inability to tackle complex and long-context tasks. Additionally, LLMs often suffer from a lack of transparency and are prone to producing hallucinations. To address these challenges, we propose \textbf{JERR}, a novel framework designed to enhance long-context comprehension via graph-based reasoning in LLMs. JERR integrates three key components: synopsis extraction, graph construction, and relational reasoning. First, synopsis is extracted by chunking text strategically, allowing the model to summarize and understand information more efficiently. Second, we build a directed acyclic graph (DAG) to resolve redundancy, ensuring logical consistency and clarity. Finally, we incorporate Monte Carlo Tree Search (MCTS) to help the model navigate complex reasoning paths, ensuring more accurate and interpretable outputs. This framework provides a novel solution that enables LLMs to handle extended contexts and complex reasoning tasks with improved reliability and transparency. Experimental results show that JERR consistently outperforms all baselines on the ROUGE and F1 metrics, achieving the highest scores on the LLM-Rater evaluation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。