构建数学推导图数据集，用以识别论文中公式间的逻辑依赖关系。
Mathematical Derivation Graphs: A Relation Extraction Task in STEM Manuscripts
- 基于107篇STEM论文，人工标注2000+公式间依赖关系
- 最佳大模型在关系抽取任务上F1仅达45%-52%
- 提出结合分析算法与模型的混合方法提升性能
近年来自然语言处理（NLP）的发展，尤其是大规模语言模型（LLMs）的兴起，显著推动了文本分析的进步。然而，将这些技术应用于数学公式及其文本内关系的分析仍效果参差。本文首次尝试将关系抽取任务拓展至理解科学、技术、工程与数学（STEM）论文中公式间的依赖关系。研究构建了数学推导图数据集（MDGD），从arXiv语料库随机采样107篇已发表的STEM论文，包含超过2000个手工标注的公式间依赖关系，形成一种称为推导图的新结构，用于总结论文的数学内容。作者系统评估了多种分析与机器学习（ML）模型在识别和提取每篇文章推导关系方面的能力，并与真实标注结果对比。结果显示，最佳测试的大型语言模型在该任务上的F1分数约为45%–52%，并尝试通过结合分析算法与其他方法来提升表现。
原文摘要 · Abstract (English)
Recent advances in natural language processing (NLP), particularly with the emergence of large language models (LLMs), have significantly enhanced the field of textual analysis. However, while these developments have yielded substantial progress in analyzing natural language text, applying analysis to mathematical equations and their relationships within texts has produced mixed results. This paper takes the initial steps in expanding the problem of relation extraction towards understanding the dependency relationships between mathematical expressions in STEM articles. The authors construct the Mathematical Derivation Graphs Dataset (MDGD), sourced from a random sampling of the arXiv corpus, containing an analysis of $107$ published STEM manuscripts with over $2000$ manually labeled inter-equation dependency relationships, resulting in a new object referred to as a derivation graph that summarizes the mathematical content of the manuscript. The authors exhaustively evaluate analytical and machine learning (ML) based models to assess their capability to identify and extract the derivation relationships for each article and compare the results with the ground truth. The authors show that the best tested LLMs achieve $F_1$ scores of $\sim45\%-52\%$, and attempt to improve their performance by combining them with analytic algorithms and other methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。