用排序机制提升大模型长文本记忆能力，效果更优。
Long Context Modeling with Ranked Memory-Augmented Retrieval
- 根据相关性动态排序记忆条目，改进信息检索
- 在标准测试中表现领先，长文本处理更高效
- 适合需要长上下文理解的场景，如文档分析
有效管理长期记忆对处理长上下文的语言模型至关重要。我们提出增强型排序记忆增强检索（ERMAR）框架，通过相关性动态排序记忆条目。不同于以往模型，ERMAR采用新颖的相关性评分机制和点式重排序模型，对键值嵌入进行优化，借鉴信息检索中的学习排序技术。结合历史使用模式与自适应检索，ERMAR在标准基准上取得当前最佳表现，展现出优异的可扩展性与长上下文任务性能。
原文摘要 · Abstract (English)
Effective long-term memory management is crucial for language models handling extended contexts. We introduce the Enhanced Ranked Memory Augmented Retrieval (ERMAR) framework, which dynamically ranks memory entries based on relevance. Unlike prior models, ERMAR employs a novel relevance scoring mechanism and a pointwise re-ranking model for key-value embeddings, inspired by learning-to-rank techniques in information retrieval. By integrating historical usage patterns and adaptive retrieval, ERMAR achieves state-of-the-art results on standard benchmarks, demonstrating superior scalability and performance in long-context tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。