用概率模型优化上下文学习的示例顺序，提升小样本准确率。
PLR: Plackett-Luce for Reordering In-Context Learning Examples
- 采用Plackett-Luce模型建模示例顺序分布，通过迭代优化高绩效顺序概率。
- 在多个分类任务上，使用4~32个示例时，少样本准确率均有提升。
- 适用于标签不可靠的数学推理任务，优于传统基于标签的排序方法。
上下文学习（ICL）通过少量示例调整大语言模型，无需参数更新。然而性能对示例顺序极为敏感，而遍历n!种顺序不可行。现有方法依赖模型置信度或直接搜索最优顺序。本文提出PLR，一种基于概率的示例排序方法：用Plackett-Luce分布建模顺序，通过任务级指标迭代优化参数，使概率集中于高绩效顺序；利用Gumbel扰动-排序法高效采样候选顺序。在多个分类基准测试中，PLR在k ∈ {4, 8, 16, 32}的少样本设置下均显著提升准确率；并在标签不可用的数学推理任务中展现优势。代码已开源。
原文摘要 · Abstract (English)
In-context learning (ICL) adapts large language models by conditioning on a small set of ICL examples, avoiding costly parameter updates. Among other factors, performance is often highly sensitive to the ordering of the examples. However, exhaustive search over the $n!$ possible orderings is infeasible. Therefore more efficient ordering methods use model confidence measures (e.g., label-probability entropy) over label sets or take a direct approach to finding the best ordering. We propose PLR, a probabilistic approach to in-context example ordering that replaces discrete ordering search with learning a probability distribution over orderings with the Plackett-Luce model. PLR models orderings using a Plackett-Luce distribution and iteratively updates its parameters to concentrate probability mass on high-performing orderings under a task-level metric. Candidate orderings are sampled efficiently via a Gumbel perturb-and-sort procedure. Experiments on multiple classification benchmarks show that PLR consistently improves few-shot accuracy for $k \in \{4, 8, 16, 32\}$ examples, and we further demonstrate gains on mathematical reasoning tasks where label-based ordering methods are not applicable. Our code is available at https://github.com/Batorskq/PLR.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。