自动从论文中提取LLM信息生成卡片,助研究者快速了解模型细节。
AutoLLM-CARD: Towards a Description and Landscape of Large Language Models
- 用命名实体和关系抽取技术自动抓取模型名、许可证、应用场景。
- 从106篇论文中提取129条带许可证关联、106条带应用关联的语句。
- 生成结构化数据支持构建大模型卡片知识图谱,适合开发者与研究者使用。
随着自然语言处理领域快速发展,各类大语言模型不断涌现以应对多样化的任务需求。随着论文数量激增,研究人员面临信息过载挑战。因此,亟需一种能自动从学术论文中提取并组织大语言模型关键信息的系统。当前标准的模型信息记录方式为大语言模型卡片(LLM-Card)。本文提出一种自动从科研论文生成LLM卡片的方法,采用命名实体识别(NER)与关系抽取(RE)技术,自动提取模型名称、许可证及应用场景等关键信息。通过构建三个词典(模型名、许可证、应用),共提取11,051条句子,经人工审核筛选出129条含模型名与许可证关联的句子,以及106条含模型名与应用关联的句子。最终构建的数据集可用于基于关系知识图谱的大语言模型卡片可视化。相关代码与成果将开源共享。
原文摘要 · Abstract (English)
With the rapid growth of the Natural Language Processing (NLP) field, a vast variety of Large Language Models (LLMs) continue to emerge for diverse NLP tasks. As more papers are published, researchers and developers face the challenge of information overload. Thus, developing a system that can automatically extract and organise key information about LLMs from academic papers is particularly important. The standard format for documenting information about LLMs is the LLM model card (\textbf{LLM-Card}). We propose a method for automatically generating LLM model cards from scientific publications. We use Named Entity Recognition (\textbf{NER}) and Relation Extraction (\textbf{RE}) methods that automatically extract key information about LLMs from the papers, helping researchers to access information about LLMs efficiently. These features include model \textit{licence}, model \textit{name}, and model \textit{application}. With these features, we can form a model card for each paper. We processed 106 academic papers by defining three dictionaries -- LLM's name, licence, and application. 11,051 sentences were extracted through dictionary lookup, and the dataset was constructed through manual review of the final selection of 129 sentences with a link between the name and the \textit{licence}, and 106 sentences with a link between the model name and the \textit{application}. The resulting resource is relevant for LLM card illustrations using relational knowledge graphs. Our code and findings can contribute to automatic LLM card generation. Data and code in \textsc{autoLLM-Card} will be shared and freely available at \url{https://github.com/shengwei-tian/dependency-parser-visualization}
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。