用大模型生成学术分类体系,让自动生成的分类更像专家写的。
TaxoAlign: Scholarly Taxonomy Generation Using Language Models
- 分三阶段构建,用指令引导生成层次化分类结构。
- 在460个真实论文分类上测试,生成结果比基线方法更接近人工分类。
- 提出新评估框架,可量化比较自动生成与人工分类的结构相似性。
分类体系在帮助研究者以层级方式组织和导航知识方面至关重要,也是撰写全面文献综述的基础。现有自动综述生成方法未将生成结果与人类专家编写的分类结构进行对比。为此,我们提出一种自动化分类体系生成方法,旨在弥合人工与自动生成分类之间的差距。为此,我们构建了CS-TaxoBench基准数据集,包含从人工撰写综述论文中提取的460个分类体系,并额外添加了80个来自会议综述论文的精选测试集。我们提出TaxoAlign方法,一种基于主题、受指令引导的三阶段学术分类生成方案。同时,我们设计了一套严格的自动化评估框架,用于衡量自动生成分类与人类专家分类在结构对齐性和语义连贯性上的差异。我们在CS-TaxoBench上对TaxoAlign及多个基线方法进行了评估,采用自动化指标和人工评估。结果表明,TaxoAlign在几乎所有指标上均显著优于基线。代码与数据可在https://github.com/AvishekLahiri/TaxoAlign获取。
原文摘要 · Abstract (English)
Taxonomies play a crucial role in helping researchers structure and navigate knowledge in a hierarchical manner. They also form an important part in the creation of comprehensive literature surveys. The existing approaches to automatic survey generation do not compare the structure of the generated surveys with those written by human experts. To address this gap, we present our own method for automated taxonomy creation that can bridge the gap between human-generated and automatically-created taxonomies. For this purpose, we create the CS-TaxoBench benchmark which consists of 460 taxonomies that have been extracted from human-written survey papers. We also include an additional test set of 80 taxonomies curated from conference survey papers. We propose TaxoAlign, a three-phase topic-based instruction-guided method for scholarly taxonomy generation. Additionally, we propose a stringent automated evaluation framework that measures the structural alignment and semantic coherence of automatically generated taxonomies in comparison to those created by human experts. We evaluate our method and various baselines on CS-TaxoBench, using both automated evaluation metrics and human evaluation studies. The results show that TaxoAlign consistently surpasses the baselines on nearly all metrics. The code and data can be found at https://github.com/AvishekLahiri/TaxoAlign.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。