通过动态协作图提升长文本理解，用小上下文实现大模型效果
Graph of Agents: Principled Long Context Modeling by Emergent Multi-Agent Collaboration
- 构建输入自适应的代理协作图，以信息论目标优化长文本处理
- 在6个文档问答任务上，相比固定结构多代理系统提升16.35%准确率
- 2K上下文胜过128K原生窗口，适合需要高效长文本处理的场景
作为一种与模型无关的长上下文建模方法，多智能体系统可在不重新训练或修改架构的前提下处理超出大语言模型上下文窗口的输入。然而，其性能通常严重依赖手工设计的多智能体协作策略和提示工程，限制了泛化能力。本文提出一个原理性框架，将模型无关的长上下文建模问题形式化为压缩问题，得到一个信息论压缩目标。基于此框架，我们提出图状智能体（GoA），动态构建依赖输入的协作结构以最大化该目标。在 Llama 3.1 8B 与 Qwen3 8B 上的六个文档问答基准测试中，GoA 分别使检索增强生成的平均 F₁ 得分提升 5.7%，比采用固定协作结构的强基线提升 16.35%。即使仅使用 2K 上下文窗口，GoA 在 LongBench 上仍超越 128K 上下文窗口的 Llama 3.1 8B，有效上下文长度显著增加。代码已开源。
原文摘要 · Abstract (English)
As a model-agnostic approach to long context modeling, multi-agent systems can process inputs longer than a large language model's context window without retraining or architectural modifications. However, their performance often heavily relies on hand-crafted multi-agent collaboration strategies and prompt engineering, which limit generalizability. In this work, we introduce a principled framework that formalizes the model-agnostic long context modeling problem as a compression problem, yielding an information-theoretic compression objective. Building on this framework, we propose Graph of Agents (GoA), which dynamically constructs an input-dependent collaboration structure that maximizes this objective. For Llama 3.1 8B and Qwen3 8B across six document question answering benchmarks, GoA improves the average $F_1$ score of retrieval-augmented generation by 5.7\% and a strong multi-agent baseline using a fixed collaboration structure by 16.35\%, respectively. Even with only a 2K context window, GoA surpasses the 128K context window Llama 3.1 8B on LongBench, showing a dramatic increase in effective context length. Our source code is available at https://github.com/tjoo512/graph-of-agents.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。