arXiv:2605.23572cs.IRcs.AI2026-05

用三阶段训练让小模型逼近大模型检索效果,大幅降低延迟。

HARNESS-LM: A Three-Phase Training Recipe for Harnessing SLMs in Sponsored Search Retrieval

论文配图:HARNESS-LM: A Three-Phase Training Recipe for Harnessing SLMs in Sponsored Search Retrieval
图 1 · 摘自论文原文
  • 分三阶段训练:先微调大模型作教师,再用L2对齐压缩小模型,最后对比学习优化
  • 在Bing广告数据上恢复98%以上大模型精度,推理延迟降低27倍,吞吐量提升20倍
  • 适合需要高吞吐、低延迟的工业级搜索推荐系统,尤其关注效果与效率平衡

在竞争激烈的赞助搜索领域,如何在检索质量与生产延迟之间取得平衡是关键挑战。尽管基于小语言模型(SLMs)如Qwen3-Embedding-4B/8B的大规模检索模型在公开基准上表现优异,但在高吞吐、低延迟的生产环境中仍难以部署。本文提出HARNESS-LM(HLM),一种三阶段训练框架,将大规模检索器的能力迁移到轻量、低成本的小模型中。方法包括:(1) 微调千亿参数级的SLM构建高性能参考(“教师”)检索器;(2) 通过L2目标对齐查询表示,将知识蒸馏至参数少于600M的学生编码器;(3) 应用最终对比学习精炼阶段以优化学生模型的检索性能。我们还进行了全面的实证研究,分析对齐目标、嵌入维度、模型规模、架构和优化策略等设计选择,识别出最适合生产环境的配置。在真实世界Bing Ads评估基准上,HLM在多个设置下恢复了超过98%的参考检索器精度,同时在线查询编码延迟降低27倍,NVIDIA A100 GPU上的吞吐量提高20倍。在Bing Ads上的在线A/B测试显示,相比当前运行的集成检索器,采用190M参数模型部署后带来+1%收入、+0.6%曝光和+0.4%点击提升,充分验证了该方案在实际赞助搜索场景中的有效性。

原文摘要 · Abstract (English)

In the competitive landscape of sponsored search, balancing retrieval quality with production latency is a critical challenge. While large retrieval models based on Small Language Models (SLMs) such as Qwen3-Embedding-4B/8B set strong upper bounds on public benchmarks, their deployment in high-throughput, latency-sensitive environments remains impractical. In this paper, we present HARNESS-LM (HLM), a three-phase training framework for transferring the capabilities of large-scale retrievers into compact, cost-efficient models. The approach comprises: (1) training a high-performance reference ("teacher") retriever by fine-tuning a billion-parameter-scale SLM; (2) aligning query representations via an L2 objective to distill knowledge into a sub-600M parameter student encoder; and (3) applying a final contrastive refinement stage to optimize the student for retrieval performance. We also present a comprehensive empirical study of key design choices, including alignment objectives, embedding dimensionality, model scale, architecture, and optimization strategies, to identify configurations that are most effective in production settings. On a real-world Bing Ads evaluation benchmark, HLM recovers over 98% of the reference retriever's precision across multiple settings, while delivering up to 27x lower online query-encoder latency and 20x higher throughput on NVIDIA A100 GPUs. Online A/B testing on Bing Ads further shows a +1% Revenue, +0.6% Impression, and +0.4% Click uplift over the current ensemble of retrievers running in production with the deployed 190M parameter model, clearly highlighting the practical efficacy of the HLM recipe in a real-world sponsored search setting.

检索模型知识蒸馏小模型广告搜索

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。