受LSTM启发的多智能体系统,提升长文本理解准确性
LSTM-MAS: A Long Short-Term Memory Inspired Multi-Agent System for Long-Context Understanding
- 用链式结构模拟LSTM门控机制,分层处理文本片段
- 在5个数据集上超越最优基线,最高提升122.19%
- 适合需要精准长文本推理的应用场景
大语言模型在处理长上下文时仍面临根本性挑战。现有单模型方法多通过缩减上下文窗口或优化注意力机制,但常带来额外计算开销或扩展受限。多智能体框架虽可缓解此问题,却易导致错误累积与幻觉传播。本文受长短期记忆网络(LSTM)启发,提出LSTM-MAS多智能体系统,模拟其分层信息流与门控记忆机制以实现长上下文理解。该系统采用链式架构,每个节点包含负责段落理解的工作者、冗余过滤的过滤器、持续纠错的裁判者及全局调控信息传播与保留的管理者,分别对应LSTM的输入门、遗忘门、恒定误差循环单元与输出门。这一设计实现了受控的信息传递与选择性长期依赖建模,有效避免错误累积与幻觉扩散。我们在多个数据集上进行了全面评估,相比此前最佳多智能体方法CoA,LSTM-MAS在Narrative QA、Qasper、HotpotQA、2WikiMQA和MuSiQue上分别提升97.97%、65.75%、122.19%、39.61%和10.80%。
原文摘要 · Abstract (English)
Effectively processing long contexts remains a fundamental yet unsolved challenge for large language models (LLMs). Existing single-LLM-based methods primarily reduce the context window or optimize the attention mechanism, but they often encounter additional computational costs or constrained expanded context length. While multi-agent-based frameworks can mitigate these limitations, they remain susceptible to the accumulation of errors and the propagation of hallucinations. In this work, we draw inspiration from the Long Short-Term Memory (LSTM) architecture to design a Multi-Agent System called LSTM-MAS, emulating LSTM's hierarchical information flow and gated memory mechanisms for long-context understanding. Specifically, LSTM-MAS organizes agents in a chained architecture, where each node comprises a worker agent for segment-level comprehension, a filter agent for redundancy reduction, a judge agent for continuous error detection, and a manager agent for globally regulates information propagation and retention, analogous to LSTM and its input gate, forget gate, constant error carousel unit, and output gate. These novel designs enable controlled information transfer and selective long-term dependency modeling across textual segments, which can effectively avoid error accumulation and hallucination propagation. We conducted an extensive evaluation of our method. Compared with the previous best multi-agent approach, CoA, our model achieves improvements of 97.97%, 65.75%, 122.19%, 39.61% and 10.80% on Narrative QA, Qasper, HotpotQA, 2WikiMQA and MuSiQue, respectively.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。