arXiv:2512.12175cs.AI2025-12AAAI

通过标签传播视角提升上下文学习中示范样本的一致性,增强模型推理效果。

Rethinking Label Consistency of In-Context Learning: An Implicit Transductive Label Propagation Perspective

  • 将上下文学习视为标签传播过程,结合贝叶斯观点建模隐含概念
  • 提出基于语义与标签信息合成数据的TopK-SD方法,显著提升标签一致性
  • 在多个基准上优于传统TopK采样,适合关注ICL机制解释的研究者

大型语言模型通过少量监督示例实现上下文学习(ICL),广泛应用于自然语言处理任务。当前方法多采用检索模型选取前K个语义最相似的示例作为示范,但未保证标签一致性。本文从贝叶斯视角和归纳传播角度重新思考ICL,将其视为一种归纳学习方法,认为相似示例可引导查询的概念,一致标签作为估计。据此建立标签传播框架,关联标签一致性与传播误差界。提出一种融合语义与标签信息的数据合成方法,采用TopK-SD采样获取具有一致标签的示范样本。实验表明,TopK-SD在多个基准上优于原始TopK采样,为理解ICL内在机制提供新视角。

原文摘要 · Abstract (English)

Large language models (LLMs) perform in-context learning (ICL) with minimal supervised examples, which benefits various natural language processing (NLP) tasks. One of the critical research focus is the selection of prompt demonstrations. Current approaches typically employ retrieval models to select the top-K most semantically similar examples as demonstrations. However, we argue that existing methods are limited since the label consistency is not guaranteed during demonstration selection. Our cognition derives from the Bayesian view of ICL and our rethinking of ICL from the transductive label propagation perspective. We treat ICL as a transductive learning method and incorporate latent concepts from Bayesian view and deduce that similar demonstrations guide the concepts of query, with consistent labels serving as estimates. Based on this understanding, we establish a label propagation framework to link label consistency with propagation error bounds. To model label consistency, we propose a data synthesis method, leveraging both semantic and label information, and use TopK sampling with Synthetic Data (TopK-SD) to acquire demonstrations with consistent labels. TopK-SD outperforms original TopK sampling on multiple benchmarks. Our work provides a new perspective for understanding the working mechanisms within ICL.

上下文学习标签一致性贝叶斯方法示范选择

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。