用自动生成的示例提升文档级信息抽取效果
DocIE@XLLM25: In-Context Learning for Information Extraction using Fully Synthetic Demonstrations
- 用大模型自动生成高质量演示样例，替代人工标注
- 在零样本场景下实现59000实体、30000关系三元组的抽取
- 适合研究零样本信息抽取与合成数据生成的学者
在零样本或少样本设置下，文档级实体与关系抽取仍缺乏大规模高质量标注语料。本文提出一种全自动、基于大语言模型的合成数据生成与上下文学习流水线。与依赖人工标注示例或直接零样本推理的方法不同，本方法结合合成数据生成与基于检索的上下文学习，采用优化推理能力的语言模型，无需人工标注即可构建高质量示例库，并在推理时动态检索相关样例。基于该方法，我们生成了一个包含超过5000篇维基百科摘要的合成数据集，涵盖约59000个实体和30000个关系三元组。最后，我们在DocIE共享任务上评估了上下文学习性能，在零样本条件下从长文档中抽取实体与关系。结果表明，即使对最先进大模型而言，文档级联合实体与关系抽取仍是极具挑战的任务。
原文摘要 · Abstract (English)
Large, high-quality annotated corpora remain scarce in document-level entity and relation extraction in zero-shot or few-shot settings. In this paper, we present a fully automatic, LLM-based pipeline for synthetic data generation and in-context learning for document-level entity and relation extraction. In contrast to existing approaches that rely on manually annotated demonstrations or direct zero-shot inference, our method combines synthetic data generation with retrieval-based in-context learning, using a reasoning-optimized language model. This allows us to build a high-quality demonstration database without manual annotation and to dynamically retrieve relevant examples at inference time. Based on our approach we produce a synthetic dataset of over $5k$ Wikipedia abstracts with approximately $59k$ entities and $30k$ relation triples. Finally, we evaluate in-context learning performance on the DocIE shared task, extracting entities and relations from long documents in a zero-shot setting. We find that in-context joint entity and relation extraction at document-level remains a challenging task, even for state-of-the-art large language models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。