arXiv:2602.12123cs.LGcs.AI2026-02

用轻量元学习方法高效选出最适合作为上下文示例的样本。

Meta-Sel: Efficient Demonstration Selection for In-Context Learning via Supervised Meta-Learning

  • 基于监督元学习构建可解释的评分函数,仅需两个简单特征。
  • 在4个数据集、5个模型上表现稳定,小模型上效果更显著。
  • 无需微调或额外调用大模型,适合实时查询场景。

演示选择是上下文学习中的实际瓶颈:在有限提示预算下,准确率会因选取的少样本示例不同而大幅波动,但选择过程必须足够廉价,以支持对大规模候选池的每查询处理。本文提出 Meta-Sel,一种轻量级监督元学习方法,用于意图分类任务中演示样本的选择。该方法通过从训练集中采样(候选,查询)对,并利用类别一致性作为监督信号构建元数据集,随后在两个低成本元特征上训练一个校准的逻辑回归器:TF-IDF余弦相似度和长度兼容性比率。推理时,选择器对整个候选池执行一次向量化评分,返回前k个演示样本,无需模型微调、无需在线探索,也无需额外的大模型调用。这实现了确定性排序,使选择机制可通过可解释的特征权重轻松审计。除了提出 Meta-Sel,本文还对演示选择进行了广泛实证研究,对比了12种方法——涵盖提示工程基线、启发式选择、强化学习和基于影响的方法——在四个意图数据集和五个开源大模型上的表现。实验表明,Meta-Sel 在所有基准测试中始终位列前列,尤其在小模型上表现突出,其选择质量可部分弥补模型容量不足的问题,且保持较低的运行开销。

原文摘要 · Abstract (English)

Demonstration selection is a practical bottleneck in in-context learning (ICL): under a tight prompt budget, accuracy can change substantially depending on which few-shot examples are included, yet selection must remain cheap enough to run per query over large candidate pools. We propose Meta-Sel, a lightweight supervised meta-learning approach for intent classification that learns a fast, interpretable scoring function for (candidate, query) pairs from labeled training data. Meta-Sel constructs a meta-dataset by sampling pairs from the training split and using class agreement as supervision, then trains a calibrated logistic regressor on two inexpensive meta-features: TF--IDF cosine similarity and a length-compatibility ratio. At inference time, the selector performs a single vectorized scoring pass over the full candidate pool and returns the top-k demonstrations, requiring no model fine-tuning, no online exploration, and no additional LLM calls. This yields deterministic rankings and makes the selection mechanism straightforward to audit via interpretable feature weights. Beyond proposing Meta-Sel, we provide a broad empirical study of demonstration selection, benchmarking 12 methods -- spanning prompt engineering baselines, heuristic selection, reinforcement learning, and influence-based approaches -- across four intent datasets and five open-source LLMs. Across this benchmark, Meta-Sel consistently ranks among the top-performing methods, is particularly effective for smaller models where selection quality can partially compensate for limited model capacity, and maintains competitive selection-time overhead.

上下文学习演示选择元学习轻量化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。