优化提问方式比调大模型更能提升信息抽取效果。
Improving Information Extraction with Learned Queries
- 用可学习的问答生成与反馈优化方法改进提问设计。
- 在临床数据集上提升18.6点F1，超过模型规模扩大效果。
- 适合希望低成本提升抽取性能的研究者和应用开发者。
当信息抽取失败时，人们通常倾向于改进模型本身，例如通过扩大规模或优化推理。本文表明，流水线中另一关键环节——用于提取信息的查询设计——同样重要。在四个临床基准和五种大语言模型上，仅优化问题设计即可使性能提升18.6个F1分数点，超过使用更大模型的效果。为此，我们提出List of Questions（LoQ），生成文档相关的问答集合，并引入FeedQ，一种基于抽取结果反馈的迭代优化方法。经优化后的问答可用于训练轻量级生成器：微调后40亿参数模型的表现达到甚至超越人工专家设计的基线，显著优于未调优的大模型。我们发布包含12,820个优化问答的数据集，推动信息抽取研究将问题设计视为核心任务。
原文摘要 · Abstract (English)
When information extraction fails, a natural instinct is to improve the model doing it: for example, by scaling it up or refining its reasoning. In this paper, we show that another part of the pipeline matters at least as much: the queries used to elicit this information. Across four clinical benchmarks and five LLMs, improving the question design alone raises performance by 18.6 F1-score points, i.e. more than using larger extraction models. To make such question design learnable, we introduce List of Questions (LoQ), which generates document-specific question sets, and FeedQ, a feedback-driven optimization method that iteratively refines questions against extraction outcomes. The resulting optimized questions can be used to train lightweight generators: with fine-tuning, 4B-parameter models match or outperform expert-derived baselines and substantially exceed the performance of much larger untuned models. We release a dataset of 12,820 optimized questions to support a broader shift in information extraction research toward treating question design as a first-class problem.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。