用生成式检索提升书籍搜索精准度,保留章节结构信息。
Generative Retrieval for Book search
- 基于书本大纲构建多层级数据增强,保持内容层次
- 提出双层位置编码与记忆注意力,支持长文本上下文建模
- 在百度数据集上相较领先方法提升9.8%的排序效果
在书籍搜索中,需根据查询返回相关书籍信息。书籍包含元数据、目录和正文等复杂多维信息,其中目录提供了章节间的层级结构。生成式检索(GR)是一种将语料库信息整合至单一模型以生成相关文档标识符的新范式。如何将其应用于书籍搜索?由于书籍具有独特性:模型需保留复杂多维信息,增加了对标注数据的需求;若将书籍信息拆分为独立片段学习,则可能损失层级结构。本文提出一种面向书籍搜索的生成式检索框架(GBS),包含两大核心组件:数据增强与基于目录的书籍编码。数据增强方面,GBS构建多个查询-书籍配对用于训练,基于目录、不同形式的书籍内容生成多种书籍标识符,并模拟真实检索场景的伪查询。包括促进覆盖度的标识符增强,使模型学会有效索引;以及增强多样性的查询增强,使模型学会有效检索。基于目录的书籍编码通过双层位置编码与保留注意力机制,提升长序列的外推能力,维持上下文连贯性。在百度自有的数据集上的实验表明,GBS优于强基线,在MRR@20上比当前最优的RIPOR方法提升9.8%。
原文摘要 · Abstract (English)
In book search, relevant book information should be returned in response to a query. Books contain complex, multi-faceted information such as metadata, outlines, and main text, where the outline provides hierarchical information between chapters and sections. Generative retrieval (GR) is a new retrieval paradigm that consolidates corpus information into a single model to generate identifiers of documents that are relevant to a given query. How can GR be applied to book search? Directly applying GR to book search is a challenge due to the unique characteristics of book search: The model needs to retain the complex, multi-faceted information of the book, which increases the demand for labeled data. Splitting book information and treating it as a collection of separate segments for learning might result in a loss of hierarchical information. We propose an effective Generative retrieval framework for Book Search (GBS) that features two main components: data augmentation and outline-oriented book encoding. For data augmentation, GBS constructs multiple query-book pairs for training; it constructs multiple book identifiers based on the outline, various forms of book contents, and simulates real book retrieval scenarios with varied pseudo-queries. This includes coverage-promoting book identifier augmentation, allowing the model to learn to index effectively, and diversity-enhanced query augmentation, allowing the model to learn to retrieve effectively. Outline-oriented book encoding improves length extrapolation through bi-level positional encoding and retentive attention mechanisms to maintain context over long sequences. Experiments on a proprietary Baidu dataset demonstrate that GBS outperforms strong baselines, achieving a 9.8\% improvement in terms of MRR@20, over the state-of-the-art RIPOR method...
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。