用未见簇数选择演示样本,提升大模型上下文学习效果
UCS: Estimating Unseen Coverage for Improved In-Context Learning

- 基于模型一致嵌入发现潜在簇,估计候选集未暴露的簇数
- 在多个任务上提升准确率2-6%,相同预算下优于现有方法
- 无需训练,可与各类选择器结合,适合研究模型内在结构
上下文学习性能高度依赖提示中演示样本的选择,但现有选择器多基于相关性或多样性等启发式策略,难以提供演示集覆盖率的洞察。本文提出无训练的未见覆盖率选择(UCS),其核心思想是:优质演示集应揭示当前已选子集未暴露的潜在簇。UCS通过(1)从模型一致嵌入中诱导离散潜在簇,(2)利用平滑古德-图灵估计算法,根据经验频率谱估计候选子集中未揭示的簇数。相比以往方法,UCS为覆盖率导向且无需训练,可通过简单正则化目标无缝集成到查询相关和查询无关的选择基线中。在多个意图分类与推理基准上使用前沿大语言模型的实验表明,将UCS融入强基线后,在相同选择预算下,可稳定提升上下文学习准确率2%-6%,同时揭示任务与模型层面的潜在簇分布。代码已开源于https://github.com/Raina-Xin/UCS。
原文摘要 · Abstract (English)
In-context learning (ICL) performance depends critically on which demonstrations are placed in the prompt, yet most existing selectors prioritize heuristic notions of relevance or diversity and provide limited insight into the coverage of a demonstration set. We propose Unseen Coverage Selection (UKS), a training-free, subset-level coverage prior motivated by the principle that a good demonstration set should expose the model to latent cluster unrevealed by the currently selected subset. UCS operationalizes this idea by (1) inducing discrete latent clusters from model-consistent embeddings and (2) estimating the number of unrevealed clusters within a candidate subset via a Smoothed Good--Turing estimator from its empirical frequency spectrum. Unlike previous selection methods, UCS is coverage-based and training-free, and can be seamlessly combined with both query-dependent and query-independent selection baselines via a simple regularized objective. Experiments on multiple intent-classification and reasoning benchmarks with frontier Large Language Models show that augmenting strong baselines with UCS consistently improves ICL accuracy by up to 2-6% under the same selection budget, while also yielding insights into task- and model-level latent cluster distributions. Code is available at https://github.com/Raina-Xin/UCS.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。