arXiv:2603.09835cs.CL2026-03

用概率图优化分块顺序,提升长文本推理准确率。

Chow-Liu Ordering for Long-Context Reasoning in Chain-of-Agents

  • 基于Chow-Liu树构建分块依赖关系,优先处理强相关片段。
  • 在三个基准上,问答相关性和精确匹配率均优于默认和语义排序。
  • 适合需要高效长文本推理的智能体系统设计者使用。

Chain-of-Agents(CoA)通过将长文本输入切分为块,由基于大模型的工作者代理按序处理并共享有限记忆来应对长上下文问题。从概率角度看,其目标是近似能联合处理全上下文的模型所对应的条件分布。该方法通过潜在状态因子分解,仅传递先前证据的有限摘要,形成信息瓶颈,导致最终证据状态对处理顺序敏感。本文研究长上下文推理中的分块顺序问题,采用经典的Chow-Liu树学习块间依赖结构,优先处理强相关块。实验证明,基于该树的广度优先遍历生成的顺序可减少代理间的信息损失,在三个长上下文基准上,显著优于默认文档分块顺序与基于语义得分的排序,提升答案相关性与精确匹配率。

原文摘要 · Abstract (English)

Sequential multi-agent reasoning frameworks such as Chain-of-Agents (CoA) handle long-context queries by decomposing inputs into chunks and processing them sequentially using LLM-based worker agents that read from and update a bounded shared memory. From a probabilistic perspective, CoA aims to approximate the conditional distribution corresponding to a model capable of jointly reasoning over the entire long context. CoA achieves this through a latent-state factorization in which only bounded summaries of previously processed evidence are passed between agents. The resulting bounded-memory approximation introduces a lossy information bottleneck, making the final evidence state inherently dependent on the order in which chunks are processed. In this work, we study the problem of chunk ordering for long-context reasoning. We use the well-known Chow-Liu trees to learn a dependency structure that prioritizes strongly related chunks. Empirically, we show that a breadth-first traversal of the resulting tree yields chunk orderings that reduce information loss across agents and consistently outperform both default document-chunk ordering and semantic score-based ordering in answer relevance and exact-match accuracy across three long-context benchmarks.

多智能体长文本推理概率图模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。