统一抽象让检索模型一键适配多种大小和速度需求。
Tevatron-Elastic: A Unified Abstraction for Training Elastic Retrievers and Rerankers

- 用单一接口统一控制层数、令牌数和嵌入长度,灵活压缩模型。
- 训练一个检查点即可支持20种不同规模,性能曲线平滑无断层。
- 适合需要弹性部署的工业级检索系统,兼容主流模型结构。
单一模型规模难以兼顾生产环境多样性:不同场景对速度、索引大小的需求各异,最佳权衡随负载变化。在信息检索中,基于Transformer的模型可通过三种方式变小——减少层数、减少高层传递的令牌数、或生成更短嵌入,每种方法节省不同计算资源。以往研究各自独立,需不同代码与训练流程,难以组合或迁移。本文提出统一抽象,仅用一个对象表示模型可运行的所有尺寸,通过简短调度表定义训练规模。训练生成一个可服务所有尺寸的检查点,部署时用户自由选择。该抽象同时覆盖检索器与重排序器,支持编码器与解码器模型,基于Hugging Face已有的接口实现,新骨干网络仅需配置变更。此前方法如马特罗什卡嵌入、提前退出、二维马特罗什卡(如Starbucks)、逐层令牌压缩,皆为本框架的特例。同一接口还支持马特罗什卡-令牌压缩(MLTC),在单个检查点中联合训练多种压缩比。验证中,我们在三个骨干网络、两个任务上训练20个检查点,质量曲线平滑,单检查点成本仅略高于单一尺寸模型,受控实验确认实际加速效果。我们开源框架与全部检查点,助力构建弹性检索系统。
原文摘要 · Abstract (English)
A single model scale challenges the flexibility of a production retrieval system: some settings need it faster, others need a smaller index, and the right trade-off changes with the workload. In the context of information retrieval (IR), a transformer-based model can be made smaller in three ways---using fewer layers, passing fewer tokens through the upper layers, or producing a shorter embedding---and each way saves a different compute resource. These options have been studied one at a time, each as its own method with its own code and training setup, which makes them hard to combine or adapt to a new model. We present~\ours to bring all three under one simple abstraction: a single object names any size the model can run at, and a short schedule lists the sizes to train. Training then produces one checkpoint that serves all of those sizes, and at deployment the user picks any of them. The same abstraction covers both retrievers and rerankers and both encoder and decoder models, as it works through interfaces that Hugging Face transformers already expose; a new backbone is a configuration change, not new modeling code. Prior methods---Matryoshka embeddings, early exit, 2D~Matryoshka (e.g., Starbucks), and layerwise token compression---become special cases of our unified abstraction. The same interface also enables Matryoshka~LTC (MLTC), which jointly trains several token-compression ratios in one retriever checkpoint. To validate our framework, we train 20 checkpoints across three backbones and two tasks: the quality curves are smooth, one checkpoint costs little over a model trained for a single size, and a controlled study confirms the wallclock speedups. We release the framework and all checkpoints as a resource for building elastic retrieval systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。