通过引导空间变换，让预计算的文本嵌入实时响应指令，速度提升6~300倍。
Don't Reinvent the Wheel: Efficient Instruction-Following Text Embedding based on Guided Space Transformation
- 用轻量级变换机制动态调整预计算嵌入，无需重编码全文。
- 在9个真实数据集上，性能超越现有方法，处理速度提升6~300倍。
- 适合需要快速响应用户指令的场景，如智能搜索、对话系统。
本文研究指令跟随文本嵌入任务，旨在生成能随用户指令动态调整、突出文本特定属性的嵌入表示。现有方法因需为每条新指令重新编码整个语料库，导致显著计算开销。为此，我们提出GSTransform框架，基于引导空间变换实现高效指令跟随。核心观察是：指令相关信息已隐含于通用嵌入中但未被充分利用。与重复编码不同，GSTransform通过少量指令标注文本数据，实时引导预计算嵌入的轻量级变换，使其对齐用户指令。在九个真实数据集上的三个指令感知下游任务中，实验表明该方法在保持更优性能的同时，实现了6~300倍的实时处理加速。代码开源：https://github.com/YingchaojieFeng/GSTransform。
原文摘要 · Abstract (English)
In this work, we investigate an important task named instruction-following text embedding, which generates dynamic text embeddings that adapt to user instructions, highlighting specific attributes of text. Despite recent advancements, existing approaches suffer from significant computational overhead, as they require re-encoding the entire corpus for each new instruction. To address this challenge, we propose GSTransform, a novel instruction-following text embedding framework based on Guided Space Transformation. Our key observation is that instruction-relevant information is inherently encoded in generic embeddings but remains underutilized. Instead of repeatedly encoding the corpus for each instruction, GSTransform is a lightweight transformation mechanism that adapts pre-computed embeddings in real time to align with user instructions, guided by a small amount of text data with instruction-focused label annotation. We conduct extensive experiments on three instruction-awareness downstream tasks across nine real-world datasets, demonstrating that GSTransform improves instruction-following text embedding quality over state-of-the-art methods while achieving dramatic speedups of 6~300x in real-time processing on large-scale datasets. The source code is available at https://github.com/YingchaojieFeng/GSTransform.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。