小模型也能靠推理时计算提升检索效果,无需训练新参数。
Test-Time Compute for Frozen Embedding Models through Agentic Program Search

- 用大模型写程序自动探索144种检索策略,找到12个最优方案。
- 在14个任务上全部提升nDCG@10,最高可实现14.7倍算力换质量。
- 方法通用性强,跨数据集和编码器仍有效,适合无标注场景。
测试时计算通常被认为只对大型推理模型有益,而小型嵌入模型则被忽视。我们提出相反观点:现代小型嵌入模型源自大型语言模型,具备潜在的测试时计算能力。研究问题在于,仅通过推理阶段的计算,不依赖辅助模型或训练参数,冻结的嵌入模型能提升多少检索质量?通过一个由大语言模型驱动的智能体循环,针对冻结编码器接口编写程序,探索了144个候选方案,生成12个帕累托最优程序,其在成本比从1.2到14.7之间权衡推理开销与质量,每个均在14个发现任务上提升nDCG@10。这些程序不使用可训练参数,恢复了倒数排名融合、Fisher线性判别、Rocchio伪相关反馈及句级MaxSim等经典检索机制。在19个未见任务及3种未见过的编码器家族上,单个固定程序在多数任务中表现更优,中位ΔnDCG@10为正,且在成本≥4时胜率高达54%至57%,尤其在从未参与发现的编码器上增益最大。相比之下,同任务训练的匹配预算学习投影头无法迁移,仅在域内提升+0.20至+0.25 nDCG@10,但在所有外部编码器上均低于基线。因此,小型嵌入模型确实继承了可用的测试时计算潜力,而冻结编码器可通过推理开销转化为可迁移的检索性能提升,无需领域标签。
原文摘要 · Abstract (English)
Test-time compute is widely believed to benefit only large reasoning models, leaving small models with nothing to gain. We argue the opposite for dense retrieval, since modern small embedding models are distilled or adapted from large language model backbones and can inherit their latent test-time-compute potential. We ask how much retrieval quality a frozen embedding model gains at inference alone, with no auxiliary model and no parameters trained at deployment. An agentic loop in which a large language model writes programs over a frozen encoder API explores 144 candidates and yields twelve Pareto-optimal programs that trade inference compute for quality across cost ratios from $c{=}1.2$ to $14.7$, every one improving nDCG@10 on all 14 discovery tasks. The programs use no trainable parameters and recover classical retrieval primitives, among them reciprocal rank fusion, the Fisher linear discriminant, Rocchio pseudo-relevance feedback, and sentence-level MaxSim. Applied unmodified to nineteen held-out tasks and three unseen encoder families, a single fixed program improves the majority of tasks, with a positive median $Δ$nDCG@10 and a 54 to 57% win-rate at $c{\ge}4$, and the gains are largest on encoder families never seen during discovery. A matched-budget learned projection head trained on the same tasks does not transfer this way, improving in-domain retrieval by $+0.20$ to $+0.25$ nDCG@10 yet falling below baseline on every held-out encoder. Small embedding models therefore inherit usable test-time-compute potential, and a frozen encoder converts inference compute into retrieval gains that transfer to new corpora and encoders with no per-domain labels.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。