arXiv:2511.00489cs.CL2025-11EMNLP被引 2

用树形结构分步推理,让大模型更懂长文本逻辑。

ToM: Leveraging Tree-oriented MapReduce for Long-Context Reasoning in Large Language Models

  • 构建文档树,自底向上递归推理
  • 在700亿参数模型上超越现有方法
  • 适合需要深度长文理解的场景

大语言模型受限于上下文窗口,在处理长文本时性能显著下降。检索增强生成(RAG)虽能分块推理,但依赖相似性排序易破坏逻辑连贯性;分割-合并框架(DCF)虽提升局部推理效率,却难以捕捉长程依赖,且孤立处理片段可能引发矛盾。为此,我们提出ToM——一种面向长文本推理的树形MapReduce框架。ToM通过层次化语义解析构建文档树(DocTree),采用自底向上的聚合策略:在映射阶段,子节点生成推理理由;在归约阶段,兄弟节点理由在父节点汇聚,解决冲突或达成共识。在700亿以上参数的大模型上实验表明,ToM显著优于现有分割-合并与检索增强方法,实现更高逻辑连贯性与长文本推理能力。代码已开源:https://github.com/gjn12-31/ToM。

原文摘要 · Abstract (English)

Large Language Models (LLMs), constrained by limited context windows, often face significant performance degradation when reasoning over long contexts. To address this, Retrieval-Augmented Generation (RAG) retrieves and reasons over chunks but frequently sacrifices logical coherence due to its reliance on similarity-based rankings. Similarly, divide-and-conquer frameworks (DCF) split documents into small chunks for independent reasoning and aggregation. While effective for local reasoning, DCF struggles to capture long-range dependencies and risks inducing conflicts by processing chunks in isolation. To overcome these limitations, we propose ToM, a novel Tree-oriented MapReduce framework for long-context reasoning. ToM leverages the inherent hierarchical structure of long documents (e.g., main headings and subheadings) by constructing a DocTree through hierarchical semantic parsing and performing bottom-up aggregation. Using a Tree MapReduce approach, ToM enables recursive reasoning: in the Map step, rationales are generated at child nodes; in the Reduce step, these rationales are aggregated across sibling nodes to resolve conflicts or reach consensus at parent nodes. Experimental results on 70B+ LLMs show that ToM significantly outperforms existing divide-and-conquer frameworks and retrieval-augmented generation methods, achieving better logical coherence and long-context reasoning. Our code is available at https://github.com/gjn12-31/ToM .

长文本推理树结构大模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。