arXiv:2603.07271cs.IR2026-03

自动发现新论文中的数据集,让研究者秒找最新资源

AutoDataset: A Lightweight System for Continuous Dataset Discovery and Search

  • 从arXiv论文标题摘要中自动识别数据集发布文章
  • 实现80%的发现效率提升,11毫秒内完成一篇论文筛选
  • 适合赶进度的研究者和需要快速找数据的团队

任务专用数据集的持续增长已成为机器学习进步的主要驱动力。然而,现有平台依赖人工整理或社区投稿,导致覆盖范围有限且更新滞后。为此,我们提出AutoDataset——一个轻量级、全自动的实时数据集发现与检索系统。该系统采用论文优先策略,持续监控arXiv,直接从新发表论文中提取数据集信息。其多阶段轻量级流水线包括:使用轻量分类器快速过滤标题与摘要,F1得分0.94,推理延迟仅11毫秒;对目标论文用GROBID解析PDF,并通过句级提取器获取数据集描述;若文本中无链接,则自动回退至LaTeX源码分析以提取数据集地址;最终将结构化记录索引至密集语义检索器,支持低延迟自然语言搜索。系统已部署为在线服务,持续接收新论文,显著缩短研究者定位新数据集的时间,实际发现效率最高提升80%。

原文摘要 · Abstract (English)

The continuous expansion of task-specific datasets has become a major driver of progress in machine learning. However, discovering newly released datasets remains difficult, as existing platforms largely depend on manual curation or community submissions, leading to limited coverage and substantial delays. To address this challenge, we introduce AutoDataset, a lightweight, automated system for real-time dataset discovery and retrieval. AutoDataset adopts a paper-first approach by continuously monitoring arXiv to detect and index datasets directly from newly published research. The system operates through a low-overhead multi-stage pipeline. First, a lightweight classifier rapidly filters titles and abstracts to identify papers releasing datasets, achieving an F1 score of 0.94 with an inference latency of 11 ms. For identified papers, we parse PDFs with GROBID and apply a sentence-level extractor to extract dataset descriptions. Dataset URLs are extracted from the paper text with an automated fallback to LaTeX source analysis when needed. Finally, the structured records are indexed using a dense semantic retriever, enabling low-latency natural language search. We deploy AutoDataset as a live system that continuously ingests new papers and provides up-to-date dataset discovery. In practice, it has been shown to significantly reduce the time required for researchers to locate newly released datasets, improving dataset discovery efficiency by up to 80%.

数据集发现自动化arXiv挖掘

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。