用情景记忆增强注意力，让大模型更准地处理超长文本。
EpMAN: Episodic Memory AttentioN for Generalizing to Longer Contexts
- 引入情景记忆模块，动态聚焦语义相关上下文片段。
- 在16k至256k token长文本上，召回与问答性能优于基线模型。
- 适合需要长文本理解的场景，如法律、医疗文档分析。
大语言模型在诸多语言任务中取得显著进展，但高效处理长上下文仍是重大挑战。本文提出EpMAN——一种基于情景记忆模块的方法，在整体关注语义相关上下文片段的同时，实现对长上下文的高效处理。通过情景注意力输出重加权解码器对存储的键值缓存（KV cache）进行自注意力计算，该方法在训练与生成阶段均有效提升性能。使用EpMAN训练的解码器在多个单跳长上下文回忆与问答基准测试中表现更强，且在16k至256k token范围内展现出更高鲁棒性，优于仅采用自注意力的基线模型及流行的检索增强生成框架。
原文摘要 · Abstract (English)
Recent advances in Large Language Models (LLMs) have yielded impressive successes on many language tasks. However, efficient processing of long contexts using LLMs remains a significant challenge. We introduce \textbf{EpMAN} -- a method for processing long contexts in an \textit{episodic memory} module while \textit{holistically attending to} semantically relevant context chunks. The output of \textit{episodic attention} is then used to reweigh the decoder's self-attention to the stored KV cache of the context during training and generation. When an LLM decoder is trained using \textbf{EpMAN}, its performance on multiple challenging single-hop long-context recall and question-answering benchmarks is found to be stronger and more robust across the range from 16k to 256k tokens than baseline decoders trained with self-attention, and popular retrieval-augmented generation frameworks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。