提升长文档检索效果，通过结构与元素感知学习增强语义理解。
SEAL: Structure and Element Aware Learning to Improve Long Structured Document Retrieval
- 引入结构感知学习和掩码元素对齐，捕捉文档层级语义。
- 在BGE-M3上将NDCG@10从73.96%提升至77.84%。
- 开源含丰富结构标注的数据集，适合工业级文档检索研究。
在长结构化文档检索中，现有方法通常在缺乏显式结构信息的数据集上微调预训练语言模型（PLMs），存在两大问题：一是未能有效利用结构特征和元素级语义，二是缺少包含结构元数据的数据集。为此，我们提出SEAL，一种新型对比学习框架，通过结构感知学习保留语义层次，并采用掩码元素对齐实现细粒度语义区分。同时，我们发布SEAL-Dataset，一个带有丰富结构标注的长结构化文档检索数据集。在自研及工业数据集上，基于多种现代PLMs的大量实验，以及在线A/B测试均表明性能持续提升，尤其在BGE-M3上，NDCG@10从73.96%提升至77.84%。相关资源已公开于https://github.com/xinhaoH/SEAL。
原文摘要 · Abstract (English)
In long structured document retrieval, existing methods typically fine-tune pre-trained language models (PLMs) using contrastive learning on datasets lacking explicit structural information. This practice suffers from two critical issues: 1) current methods fail to leverage structural features and element-level semantics effectively, and 2) the lack of datasets containing structural metadata. To bridge these gaps, we propose \our, a novel contrastive learning framework. It leverages structure-aware learning to preserve semantic hierarchies and masked element alignment for fine-grained semantic discrimination. Furthermore, we release \dataset, a long structured document retrieval dataset with rich structural annotations. Extensive experiments on both released and industrial datasets across various modern PLMs, along with online A/B testing, demonstrate consistent performance improvements, boosting NDCG@10 from 73.96\% to 77.84\% on BGE-M3. The resources are available at https://github.com/xinhaoH/SEAL.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。