用分块嵌入提升长文本检索精度，兼顾效率与可解释性。
Improving Long-Context Retrieval with Multi-Prefix Embedding
- 将文档按句末标记分块，单次前向传播生成多段嵌入
- 在多个数据集上超越或媲美现有方法，支持精准证据定位
- 仅需文档级标签训练，适合需要溯源的长文本应用
长文本检索面临矛盾：单向量嵌入丢失细节，逐词多向量方法存储开销大。我们提出多前缀嵌入（MPE），将文档按结束符（EOS）分块，通过一次因果前向传播编码全文，在每个分块边界提取一个嵌入。MPE保留跨块上下文，支持分块级最大相似度匹配，并仅需文档级相关性标签进行训练。在MLDR-en、BrowseComp-Plus和LongEmbed数据集上的实验表明，MPE在性能上可媲美或优于单向量、独立分块及多向量基线方法，同时提供自然的证据块溯源机制。
原文摘要 · Abstract (English)
Long-context retrieval exposes a tension: single-vector embeddings lose fine-grained detail, while token-level multi-vector methods incur prohibitive storage. We propose Multi-Prefix Embedding (MPE), which partitions a document into chunks separated by EOS tokens, encodes the full sequence in a single causal forward pass, and extracts one embedding at each prefix boundary. MPE retains cross-chunk context, enables chunk-level MaxSim matching, and trains with only document-level relevance labels. Experiments on MLDR-en, BrowseComp-Plus, and LongEmbed show that MPE is competitive with or outperforms single-vector, independent-chunk, and multi-vector baselines, while providing a natural source attribution mechanism for locating evidence chunks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。