构建科学文献灵感检索基准，支持三种创新思路的精准匹配。
RATIO: A Benchmark for Retrieval Across Typed Ideation Operations in Scientific Literature
- 定义三种创新操作：解决问题、拓宽视角、具体化方案。
- 基于百万篇论文构建，支持跨抽象层级的检索。
- 适合研究科学启发式生成与智能文献检索的学者。
科学文献的检索可为人类和人工智能科学家提供灵感。这种灵感有多种形式：先前工作可能直接给出问题解决方案，或在不同抽象层次上揭示方向——宏观拓展或微观实现。我们提出RATIO（Retrieval Across Typed Ideation Operations），一个大规模基准，其相关性由三类操作定义，称为‘创新操作’：Address（针对问题）获取潜在解决方法，Broaden（拓宽）获取更一般的表述，Specify（具体化）获取具体的实现案例。RATIO基于数百万篇计算机科学领域全文论文构建，采用扩展的论述标记远监督方法（此前仅用于分类），并结合大模型与人工严格验证。实验表明，针对特定操作的微调显著提升检索器性能，但仍存在巨大改进空间。RATIO为基于文献的创新支持组件提供了可扩展的训练与评估框架，开启了科学灵感检索的新研究方向。
原文摘要 · Abstract (English)
Retrieved scientific literature can serve as inspiration for both human and AI scientists. Inspiration can take different forms: prior work may directly suggest how to address a problem, or surface directions at different levels of abstraction - zooming out to a more general view or zooming in to a concrete realization. We introduce RATIO (Retrieval Across Typed Ideation Operations), a large-scale benchmark in which relevance is defined by three operations which we name ideation moves: Address retrieves potential approaches for stated problems, Broaden retrieves more general formulations, and Specify retrieves concrete instantiations. RATIO is constructed from millions of full-text scientific papers across CS literature via a general recipe that extends discourse-marker distant supervision - previously used only for classification - to corpus-scale retrieval, combined with extensive LLM and human vetting. Experiments show that operation-specific fine-tuning substantially boosts retrievers but leaves much room for further improvements. RATIO provides a scalable training and evaluation framework for retrieval components that support literature-grounded ideation, opening up new research avenues on scientific inspiration retrieval.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。