提升大模型少样本学习效果，关键在选多样例而非仅相似例
The Role of Diversity in In-Context Learning for Large Language Models
- 用多样性引导选择上下文示例，突破传统只选相似例的局限
- 在数学和代码任务上性能提升显著，对分布外查询更鲁棒
- 提出理论框架解释多样性为何能改善少样本学习
当前大语言模型的上下文学习能力至关重要，示例选择直接影响性能。尽管现有方法多聚焦于选取与查询最相似的示例，但示例多样性的作用仍未被充分探索。本文通过在情感分类、数学和代码等多样化任务上的实验，系统研究了多样性在上下文示例选择中的作用。基于 Llama-3.1、Gemma-2 与 Mistral-v0.3 系列模型的实验表明，引入多样性感知的选择方法可显著提升复杂任务（如数学与代码）的性能，并增强对分布外查询的鲁棒性。为支持该发现，我们提出了一个理论框架，用于解释多样性在上下文学习中带来的优势。
原文摘要 · Abstract (English)
In-context learning (ICL) is a crucial capability of current large language models (LLMs), where the selection of examples plays a key role in performance. While most existing approaches focus on selecting the most similar examples to the query, the impact of diversity in example selection remains underexplored. We systematically investigate the role of diversity in in-context example selection through experiments across a range of tasks, from sentiment classification to more challenging math and code problems. Experiments on Llama-3.1, Gemma-2, and Mistral-v0.3 families of models show that diversity-aware selection methods improve performance, particularly on complex tasks like math and code, and enhance robustness to out-of-distribution queries. To support these findings, we introduce a theoretical framework that explains the benefits of incorporating diversity in in-context example selection.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。