用合成数据生成领域适配嵌入，无需真实语料库或微调。
Zero-Shot Contextual Embeddings via Offline Synthetic Corpus Generation
- 仅需5个示例文档，通过分层生成数百篇合成语料。
- 零样本下在MTEB上性能接近全语料模型，误差小于0.5%。
- 适合隐私敏感或资源受限场景的高效嵌入部署。
上下文感知嵌入方法通过利用邻近文档中的语料统计特征（如词共现和主题模式）提升检索精度。然而，此类方法通常需要访问目标语料库或进行领域特定微调，在隐私敏感或资源受限场景中面临实际障碍。本文提出ZEST，一种零样本上下文自适应框架，将真实语料库替换为一次性的离线合成小型代理语料库。仅需少量代表性示例文档，通过多步分层流程生成数百篇旨在模拟关键领域分布的合成文档。推理时，冻结的上下文感知编码器使用该代理语料库，无需微调或目标语料库访问，即可生成领域适配嵌入。在MTEB基准测试中，使用五个示例文档的ZEST零样本合成上下文适配，性能与依赖完整目标语料库的模型相差不足0.5%，展现出无需重训练的显著有效性。ZEST因此为受限环境中高表现力、可适配嵌入的部署提供了实用方案。
原文摘要 · Abstract (English)
Context-aware embedding methods boost retrieval accuracy by conditioning on corpus statistics (e.g., term co-occurrence and topical patterns) extracted from neighboring documents. However, this context-aware approach requires access to the target corpus or requires domain-specific finetuning, posing practical barriers in privacy-sensitive or resource-constrained settings. We present ZEST, a zero-shot contextual adaptation framework that replaces real corpus access with a one-time offline synthesis of a compact proxy. Given only a handful exemplar documents representative of the general target domain, we use a multi-step hierarchical procedure to generate a synthetic context corpus of several hundred documents that aims to emulate key domain-specific distributions. At inference, the frozen context-aware encoder uses this proxy corpus -- without any finetuning or target corpus access -- to produce domain-adapted embeddings. Across the MTEB benchmark, ZEST's zero-shot synthetic context adaptation using only five example documents performs within 0.5% of models leveraging full target corpus access -- demonstrating remarkable efficacy without any retraining. ZEST thus provides a practical method for deploying high-performance, adaptable embeddings in constrained environments.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。