让大模型选信息时更注重多样性，提升问答和长文本摘要效果
Diversity Enhances an LLM's Performance in RAG and Long-context Task
- 用最大边缘相关和最远点采样思想优化内容选择策略
- 多样性引入使相关句子召回率显著提升
- 适合做长文本摘要、RAG问答的场景
大语言模型在快速发展的过程中，面临上下文窗口受限的挑战，主要源于自注意力机制的二次时间复杂度（$O(N^2)$，其中 $N$ 表示上下文长度）。这一限制影响了问答中的检索增强生成（RAG）和长文本摘要等任务。传统方法通常选择与查询最相似的内容，但易导致冗余并遗漏多样且相关的资讯。基于最大边缘相关（MMR）和最远点采样（FPS）的原则，我们将在内容选择中引入多样性。研究发现，融入多样性可显著提高在大模型问答与摘要前相关句子或段落的召回率。结果表明，在未来的大型语言模型应用中保持多样性对进一步提升摘要和问答表现至关重要。
原文摘要 · Abstract (English)
The rapid advancements in large language models (LLMs) have highlighted the challenge of context window limitations, primarily due to the quadratic time complexity of the self-attention mechanism (\(O(N^2)\), where \(N\) denotes the context window length). This constraint impacts tasks such as retrieval-augmented generation (RAG) in question answering (Q\&A) and long context summarization. A common approach involves selecting content with the highest similarity to the query; however, this often leads to redundancy and the exclusion of diverse yet relevant information. Building on principles from Maximal Marginal Relevance (MMR) and Farthest Point Sampling (FPS), we integrate diversity into the content selection process. Our findings reveal that incorporating diversity substantially increases the recall of selecting relevant sentences or chunks before LLM-based Q\&A and summarization. These results highlight the importance of maintaining diversity in future LLM applications to further improve summarization and Q\&A outcomes.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。