arXiv:2505.21471cs.CL2025-05ACL被引 2

通过多智能体协作突破大模型上下文限制,高效整合海量外部知识。

Scaling External Knowledge Input Beyond Context Windows of LLMs via Multi-Agent Collaboration

  • 设计多智能体框架,分布式处理超长外部知识输入。
  • 在∞Bench+等测试中,性能超越现有非训练方法,无论知识量是否超窗。
  • 高并行性保障效率,适合需大规模知识推理的应用场景。

随着后训练推理与信息检索技术的发展,大语言模型(LLMs)可融入大量检索到的知识以解决复杂任务。然而,受限于上下文窗口长度,难以扩展外部知识输入,阻碍进一步提升。现有窗口扩展方法不可避免导致信息丢失。基于LLM的多智能体方法成为处理海量输入的新范式,但现有代理编排设计存在两个核心瓶颈。本文提出多智能体框架 extbf{ ExtAgents},克服这些瓶颈,在无需长上下文训练的前提下,实现推理时知识集成的更好可扩展性。在增强的多跳问答测试集∞Bench+及其他公开数据集(如长篇综述生成)上, ExtAgents在相同外部知识输入量下显著优于现有非训练方法,无论知识量是否超出上下文窗口。同时,该方法因高并行性保持高效。我们相信,对LLM智能体协同处理更多外部知识的研究将推动实际应用发展。

原文摘要 · Abstract (English)

With the rapid advancement of post-training techniques for reasoning and information seeking, large language models (LLMs) can incorporate a large quantity of retrieved knowledge to solve complex tasks. However, the limited context window of LLMs obstructs scaling the amount of external knowledge input, prohibiting further improvement. Existing context window extension methods inevitably cause information loss. LLM-based multi-agent methods emerge as a new paradigm to handle massive input in a distributional manner, where we identify two core bottlenecks in existing agent orchestration designs. In this work, we develop a multi-agent framework, \textbf{\ExtAgents}, to overcome the bottlenecks and enable better scalability in inference-time knowledge integration without longer-context training. Benchmarked with our enhanced multi-hop question answering test, \textbf{$\boldsymbol{\infty}$Bench+}, and other public test sets including long survey generation, \ExtAgents significantly enhances the performance over existing non-training methods with the same amount of external knowledge input, regardless of whether it falls \emph{within or exceeds the context window}. Moreover, the method maintains efficiency due to high parallelism. We believe further study in the coordination of LLM agents on increasing external knowledge input could benefit real-world applications.

多智能体知识融合大模型推理

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。