让检索模型在测试时自适应优化,不改参数也能提升效果。
Test-Time Optimization of Query Embeddings with Ranking Aware Reward Maximization
- 用可学习向量存取排名奖励,无需改动模型权重
- 奖励可重复使用,提升效果最高达+8.36 nDCG@10
- 适合闭源模型、新任务或新查询的快速适配
密集检索器通过冻结编码器与预计算索引间的向量相似性进行文档排序。尽管测试时来自重排器或LLM裁判的排名奖励能提升性能,但现有方法仅使用一次即丢弃信号。更新检索器权重可使奖励复用,但需访问参数,对闭源模型不可行且计算开销大。我们提出TTT-Embed(测试时嵌入微调),将排名奖励提炼为一个轻量级、可学习的输出嵌入空间向量。该向量仅基于检索器自身候选文档的标量排名得分优化,无需模型权重、真实标签或索引修改。单个作用范围参数控制奖励复用程度(全局、任务或查询级),实现固定预算下可解释的权衡。实验显示,随着奖励预算增加,最优共享范围动态从全局转向任务级再至查询级。在五种嵌入模型和15个MTEB检索任务上,TTT-Embed使测试时检索性能最高提升+8.36 nDCG@10。关键的是,学习状态能有效泛化到未见查询(最高+8.57 nDCG@10)和未见任务(最高+4.71 nDCG@10)。此外,由于基础权重完全冻结,其成功缓解灾难性遗忘:恢复退化的通用能力(最高+8.00 nDCG@10,甚至超过原基线模型),同时保留领域内专长。这些结果确立了排名奖励作为可复用的测试时状态,实现任意嵌入模型(包括闭源API)的高效适应。
原文摘要 · Abstract (English)
Dense retrievers rank documents using vector similarity between a frozen encoder and a precomputed index. While test-time ranking rewards from a reranker or LLM judge can improve results, existing methods discard this signal after a single query. Updating the retriever's weights makes rewards reusable, but this requires parameter access, which is unavailable for closed-source models, and is computationally prohibitive. We propose TTT-Embed (Test-Time Tuning of Embeddings), a framework that distills ranking rewards into a lightweight, learned vector within the output embedding space of a frozen model. This vector is optimized purely from scalar ranking scores assigned to the retriever's own candidate documents, requiring no access to model weights, ground-truth labels, or modifications to index. A single scope parameter controls rewards reuse (global, task, or query), enabling a principled trade-off between reusability and specificity under a fixed reward computation budget. We demonstrate that as the available reward budget scales, the optimal sharing scope shifts dynamically from global-wise to task-wise and finally to query-wise. Evaluated across five embedding models and 15 MTEB retrieval tasks, TTT-Embed improves test-time retrieval by up to +8.36 nDCG@10. Crucially, the learned states generalize effectively to unseen queries (up to +8.57 nDCG@10) and unseen tasks (up to +4.71 nDCG@10). Furthermore, TTT-Embed successfully resolves catastrophic forgetting: by leaving base weights entirely frozen, it recovers degraded general capabilities (up to +8.00 nDCG@10, even surpassing the original base model) while preserving in-domain specialization. These results establish ranking rewards as a reusable test-time state, enabling budget-efficient adaptation for any embedding model, including closed-source APIs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。