用学术概念索引提升科学文献检索效果
Improving Scientific Document Retrieval with Academic Concept Index
- 构建学术概念索引,按学科分类组织论文核心概念
- 生成覆盖更广概念的查询,提升检索相关性
- 适合需要精准检索科学文献的研究者
将通用领域检索器适配到科学领域面临两大挑战:缺乏大规模领域特定的相关性标注,以及词汇与信息需求存在显著差异。现有方法虽利用大语言模型(LLMs)分别生成合成查询或辅助上下文,但忽略了科学文献中复杂的学术概念,常导致查询冗余或概念范围狭窄。为此,本文提出学术概念索引,从论文中提取关键概念,并基于学术分类体系进行组织。该索引作为基础,用于改进两个方向:一是引入概念覆盖率引导的查询生成(CCQGen),通过识别未覆盖概念动态引导LLM生成更具互补性的查询;二是提出概念聚焦的辅助上下文增强(CCExpand),利用文档片段作为对概念感知查询的简明回应。大量实验表明,融合该索引可生成更高质量的查询、实现更好的概念对齐,并显著提升检索性能。
原文摘要 · Abstract (English)
Adapting general-domain retrievers to scientific domains is challenging due to the scarcity of large-scale domain-specific relevance annotations and the substantial mismatch in vocabulary and information needs. Recent approaches address these issues through two independent directions that leverage large language models (LLMs): (1) generating synthetic queries for fine-tuning, and (2) generating auxiliary contexts to support relevance matching. However, both directions overlook the diverse academic concepts embedded within scientific documents, often producing redundant or conceptually narrow queries and contexts. To address this limitation, we introduce an academic concept index, which extracts key concepts from papers and organizes them guided by an academic taxonomy. This structured index serves as a foundation for improving both directions. First, we enhance the synthetic query generation with concept coverage-based generation (CCQGen), which adaptively conditions LLMs on uncovered concepts to generate complementary queries with broader concept coverage. Second, we strengthen the context augmentation with concept-focused auxiliary contexts (CCExpand), which leverages a set of document snippets that serve as concise responses to the concept-aware CCQGen queries. Extensive experiments show that incorporating the academic concept index into both query generation and context augmentation leads to higher-quality queries, better conceptual alignment, and improved retrieval performance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。