arXiv:2504.02871cs.CLcs.AI2025-04被引 2

用大模型自动生成医学信息抽取的标注指南,效果媲美人工且省时省力。

Synthesized Annotation Guidelines are Knowledge-Lite Boosters for Clinical Information Extraction

  • 用大模型自动合成标注指南,几乎无需人工干预。
  • 在多个临床实体识别任务上,严格F1提升最高达25.86%。
  • 生成的指南可跨任务复用,适合医疗文本挖掘研究者使用。

基于大语言模型的生成式信息抽取,尤其是少样本学习,已成为主流方法。研究表明,提供类似传统标注指南的详细、可读性强的说明能显著提升性能。但构建这些指南耗时耗力,且定义常高度依赖具体任务,难以复用。为此,本文提出一种自改进方法,利用大模型的知识归纳与文本生成能力,自动合成标注指南,几乎无需人工参与。在2012 i2b2 EVENT、2012 i2b2 TIMEX、2014 i2b2和2018 n2c2四个临床命名实体识别基准上,零样本实验显示,相比无指南基线,严格F1分别提升25.86%、4.36%、0.20%和7.75%。大模型生成的指南在多数任务中表现等同或优于人工撰写的指南,提升幅度为1.15%至4.14%。结果表明,该方法仅需极少知识与人力投入,适用于多个生物医学领域。

原文摘要 · Abstract (English)

Generative information extraction using large language models, particularly through few-shot learning, has become a popular method. Recent studies indicate that providing a detailed, human-readable guideline-similar to the annotation guidelines traditionally used for training human annotators can significantly improve performance. However, constructing these guidelines is both labor- and knowledge-intensive. Additionally, the definitions are often tailored to meet specific needs, making them highly task-specific and often non-reusable. Handling these subtle differences requires considerable effort and attention to detail. In this study, we propose a self-improving method that harvests the knowledge summarization and text generation capacity of LLMs to synthesize annotation guidelines while requiring virtually no human input. Our zero-shot experiments on the clinical named entity recognition benchmarks, 2012 i2b2 EVENT, 2012 i2b2 TIMEX, 2014 i2b2, and 2018 n2c2 showed 25.86%, 4.36%, 0.20%, and 7.75% improvements in strict F1 scores from the no-guideline baseline. The LLM-synthesized guidelines showed equivalent or better performance compared to human-written guidelines by 1.15% to 4.14% in most tasks. In conclusion, this study proposes a novel LLM self-improving method that requires minimal knowledge and human input and is applicable to multiple biomedical domains.

信息抽取医学文本大模型应用

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。