梳理信息检索模型架构演进,聚焦神经网络与大模型的变革。
A Survey of Model Architectures in Information Retrieval
- 从传统词项模型到基于Transformer的神经检索架构
- 编码器与解码器结构在零样本和复杂推理中表现突出
- 适合关注IR系统设计与大模型应用的研究者
2019年至今是信息检索(IR)与自然语言处理(NLP)领域最具颠覆性的范式转变期,自2022年起涌现出强大的大语言模型(LLMs)。基于预训练编码器架构(如BERT)及仅解码器生成式大模型的方法,显著优于早期方案,在零样本场景与复杂推理任务中表现优异。本综述聚焦信息检索中模型架构的演进,重点分析两类核心:特征提取的主干模型与相关性估计的端到端系统架构。为保持分析清晰,本文刻意区分架构设计与训练方法,专注探讨结构创新。从传统词项检索模型到现代神经方法的发展历程被系统梳理,突显Transformer架构及后续大模型发展的变革性影响。最后,展望开放挑战与新兴方向,包括架构优化以提升效率与可扩展性、多模态与多语言数据的鲁棒处理,以及面向自主搜索代理等新应用场景的适应,这些可能代表下一代信息检索范式。
原文摘要 · Abstract (English)
The period from 2019 to the present marks one of the most significant paradigm shifts in information retrieval (IR) and natural language processing (NLP), culminating in the emergence of powerful large language models (LLMs) from 2022 onward. Methods based on pretrained encoder-only architectures (e.g., BERT) as well as decoder-only generative LLMs have outperformed many earlier approaches, demonstrating particularly strong performance in zero-shot scenarios and complex reasoning tasks. This survey examines the evolution of model architectures in IR, with a focus on two key aspects: backbone models for feature extraction and end-to-end system architectures for relevance estimation. To maintain analytical clarity, we deliberately separate architectural design from training methodologies, enabling a focused examination of structural innovations in IR systems. We trace the progression from traditional term-based retrieval models to modern neural approaches, highlighting the transformative impact of transformer-based architectures and subsequent LLM developments. The survey concludes with a forward-looking discussion of open challenges and emerging research directions, including architectural optimization for efficiency and scalability, robust handling of multimodal and multilingual data, and adaptation to novel application domains such as autonomous search agents, which may represent the next paradigm in IR.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。