arXiv:2508.10036cs.CLcs.AI2025-08AAAI

让大模型自己挑最难的样本做例子,提升信息抽取准确率。

Reflect then Learn: Active Prompting for Information Extraction Guided by Introspective Confusion

  • 通过双重不确定性评分,让模型自评生成格式和内容的困惑度。
  • 在四个基准上显著优于基线,提升抽取准确率与鲁棒性。
  • 适合需要高可靠结构化生成的场景,如知识图谱构建。

大型语言模型(LLMs)在少样本信息抽取(IE)中展现巨大潜力,但其性能高度依赖于上下文示例的选择。传统选择策略常因忽视模型失误的关键来源而失效:不仅源于语义内容,还源于生成符合IE任务要求的规范格式的困难。为此,我们提出主动提示框架APIE,其核心是基于“内省困惑”原则。该方法使LLM通过双组件不确定性度量评估自身困惑,分别量化格式不确定性(生成正确语法的难度)与内容不确定性(提取语义不一致)。利用此综合评分对未标注数据排序,主动选择最困难且最具信息量的样本作为少样本示例。在四个基准上的大量实验表明,本方法持续超越强基线,显著提升抽取准确率与鲁棒性。研究强调,在构建高效可靠的结构化生成系统时,需采用细粒度、双层级的模型不确定性视角。

原文摘要 · Abstract (English)

Large Language Models (LLMs) show remarkable potential for few-shot information extraction (IE), yet their performance is highly sensitive to the choice of in-context examples. Conventional selection strategies often fail to provide informative guidance, as they overlook a key source of model fallibility: confusion stemming not just from semantic content, but also from the generation of well-structured formats required by IE tasks. To address this, we introduce Active Prompting for Information Extraction (APIE), a novel active prompting framework guided by a principle we term introspective confusion. Our method empowers an LLM to assess its own confusion through a dual-component uncertainty metric that uniquely quantifies both Format Uncertainty (difficulty in generating correct syntax) and Content Uncertainty (inconsistency in extracted semantics). By ranking unlabeled data with this comprehensive score, our framework actively selects the most challenging and informative samples to serve as few-shot exemplars. Extensive experiments on four benchmarks show that our approach consistently outperforms strong baselines, yielding significant improvements in both extraction accuracy and robustness. Our work highlights the critical importance of a fine-grained, dual-level view of model uncertainty when it comes to building effective and reliable structured generation systems.

信息抽取主动学习大模型不确定性

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。