arXiv:2502.13646cs.CL2025-02ACL被引 2

提出D.Va方法,自动筛选更有效且通用的示范样本

D.Va: Validate Your Demonstration First Before You Use It

  • 引入示范验证机制,动态评估示范样本质量
  • 在NLU和NLG任务上超越所有现有选择方法
  • 适配多种模型与检索方式,通用性强

上下文学习(ICL)在推理阶段显著提升了大语言模型(LLMs)的能力。已有研究普遍依赖直观指标选择示范样本,但该方法鲁棒性差,跨模型泛化能力弱。为此,我们提出新方法D.Va(Demonstration Validation),从示范验证角度改进示范选择。通过引入示范验证机制,D.Va能有效识别出既高效又具备高泛化能力的示范样本。实验表明,D.Va在自然语言理解(NLU)和自然语言生成(NLG)任务上全面优于现有示范选择技术。此外,该方法在不同语言模型与检索模型组合下均展现出优异的鲁棒性和泛化性能。

原文摘要 · Abstract (English)

In-context learning (ICL) has demonstrated significant potential in enhancing the capabilities of large language models (LLMs) during inference. It's well-established that ICL heavily relies on selecting effective demonstrations to generate outputs that better align with the expected results. As for demonstration selection, previous approaches have typically relied on intuitive metrics to evaluate the effectiveness of demonstrations, which often results in limited robustness and poor cross-model generalization capabilities. To tackle these challenges, we propose a novel method, \textbf{D}emonstration \textbf{VA}lidation (\textbf{D.Va}), which integrates a demonstration validation perspective into this field. By introducing the demonstration validation mechanism, our method effectively identifies demonstrations that are both effective and highly generalizable. \textbf{D.Va} surpasses all existing demonstration selection techniques across both natural language understanding (NLU) and natural language generation (NLG) tasks. Additionally, we demonstrate the robustness and generalizability of our approach across various language models with different retrieval models.

上下文学习示范选择大模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。