用大模型自动提取论文中的数据集信息，构建可查询的论文-数据集网络。
ChatPD: An LLM-driven Paper-Dataset Networking System
- 基于大模型自动识别论文中的数据集使用情况，替代人工整理。
- 在实体消歧任务中达到约90%的精确率和召回率。
- 适合研究人员快速发现任务相关数据集或相似数据集推荐。
科学研究高度依赖合适的数据集进行方法验证，但现有平台如PapersWithCode因依赖人工流程而效率低下。为此，我们提出ChatPD系统，利用大语言模型（LLM）自动化提取论文中的数据集信息，并构建结构化的论文-数据集网络。系统包含三个核心模块：论文收集、数据集信息抽取与数据集实体消歧。我们提出一种图补全与推理策略，将数据集描述映射到对应实体。大量实验表明，ChatPD在数据集使用提取上优于PapersWithCode，且在实体消歧任务中实现约90%的精度与召回率。我们已部署ChatPD持续抓取论文中的数据集使用情况，并提供任务特定数据集查询与相似数据集推荐服务。项目已开源，代码与当前论文-数据集网络可在GitHub仓库获取。
原文摘要 · Abstract (English)
Scientific research heavily depends on suitable datasets for method validation, but existing academic platforms with dataset management like PapersWithCode suffer from inefficiencies in their manual workflow. To overcome this bottleneck, we present a system, called ChatPD, that utilizes Large Language Models (LLMs) to automate dataset information extraction from academic papers and construct a structured paper-dataset network. Our system consists of three key modules: \textit{paper collection}, \textit{dataset information extraction}, and \textit{dataset entity resolution} to construct paper-dataset networks. Specifically, we propose a \textit{Graph Completion and Inference} strategy to map dataset descriptions to their corresponding entities. Through extensive experiments, we demonstrate that ChatPD not only outperforms the existing platform PapersWithCode in dataset usage extraction but also achieves about 90\% precision and recall in entity resolution tasks. Moreover, we have deployed ChatPD to continuously extract which datasets are used in papers, and provide a dataset discovery service, such as task-specific dataset queries and similar dataset recommendations. We open source ChatPD and the current paper-dataset network on this [GitHub repository]{https://github.com/ChatPD-web/ChatPD}.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。