arXiv:2412.01093cs.CLcs.IR2024-12被引 1

自动提取论文中缩写与全称配对,提升科研文本理解效率。

Automated Extraction of Acronym-Expansion Pairs from Scientific Papers

  • 结合预处理、正则表达式和大模型识别缩写及其扩展形式。
  • 仅分析缩写附近小段文本,避免误判和多义歧义。
  • 适合需要精准解析学术文献的科研人员和数据工程师。

本研究针对数字文本中广泛存在的缩写和首字母缩略词带来的挑战,提出一种新方法:通过文档预处理、正则表达式与大语言模型(GPT-4)相结合,识别缩写并映射其对应全称。正则表达式单独使用常无法准确提取扩展形式,因此本方法利用GPT-4分析缩写周围的有限文本区域,降低获得错误或多个扩展的风险。处理缩写面临诸多挑战,包括多义性、非局部性和模糊性。该方法通过自动化识别与消歧,提升了自然语言处理技术的精度与效率。研究强调了处理PDF文件的困难及文档预处理的重要性。结果表明,仅用正则表达式或仅用GPT-4均表现不佳:前者能识别缩写但受限于多样化的表达格式及作者常省略扩展;后者虽能良好获取扩展,却难以全面识别相关缩写。此外,GPT-4的随机性可能导致相同输入产生不同输出。所提算法通过预处理去除无关信息,正则表达式定位缩写,大模型辅助查找扩展,从而实现最准确一致的结果。

原文摘要 · Abstract (English)

This project addresses challenges posed by the widespread use of abbreviations and acronyms in digital texts. We propose a novel method that combines document preprocessing, regular expressions, and a large language model to identify abbreviations and map them to their corresponding expansions. The regular expressions alone are often insufficient to extract expansions, at which point our approach leverages GPT-4 to analyze the text surrounding the acronyms. By limiting the analysis to only a small portion of the surrounding text, we mitigate the risk of obtaining incorrect or multiple expansions for an acronym. There are several known challenges in processing text with acronyms, including polysemous acronyms, non-local and ambiguous acronyms. Our approach enhances the precision and efficiency of NLP techniques by addressing these issues with automated acronym identification and disambiguation. This study highlights the challenges of working with PDF files and the importance of document preprocessing. Furthermore, the results of this work show that neither regular expressions nor GPT-4 alone can perform well. Regular expressions are suitable for identifying acronyms but have limitations in finding their expansions within the paper due to a variety of formats used for expressing acronym-expansion pairs and the tendency of authors to omit expansions within the text. GPT-4, on the other hand, is an excellent tool for obtaining expansions but struggles with correctly identifying all relevant acronyms. Additionally, GPT-4 poses challenges due to its probabilistic nature, which may lead to slightly different results for the same input. Our algorithm employs preprocessing to eliminate irrelevant information from the text, regular expressions for identifying acronyms, and a large language model to help find acronym expansions to provide the most accurate and consistent results.

信息抽取NLP大模型应用

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。