提升电商搜索召回效果,解决模型升级时的性能衰减问题。
Scaling and Stabilizing Large-Scale Embedding-Based Retrieval

- 用在线跨批采样和离线混合挖掘增强负样本多样性
- 新旧模型间知识迁移使NDCG@5提升7.34%,营收增0.50%
- 适合需要稳定迭代大模型的工业级检索系统
基于嵌入的检索(EBR)是大规模电商搜索的核心,但其效果常受限于训练信号质量与编码器表征能力。标准双编码器存在训练-推理差距:训练时候选集窄,推理时需从数亿商品中区分。虽可换用更强大骨干网络缓解此问题,但直接替换成熟模型会导致检索行为不一致,丢失历史积累的领域知识。本文提出在沃尔玛部署的统一流水线,包含两项贡献:(1)混合硬负样本挖掘:结合在线跨批采样使负样本多样性提升一个数量级,以及离线混合挖掘,融合交叉编码器预测与元数据启发式规则识别细微不匹配;(2)遗产感知蒸馏:将模型从DistilBERT升级至更高容量的GTE-base编码器,引入热启动蒸馏技术,实现对旧模型领域专长的迁移。经大量离线实验与线上A/B测试验证,该方案已上线生产,实现NDCG@5提升7.34%,总营收增长0.50%。
原文摘要 · Abstract (English)
Embedding-based retrieval (EBR) is foundational to large-scale e-commerce search, yet its effectiveness is often constrained by the quality of training signals and the representational capacity of the encoder. Standard dual-encoders suffer from a training-inference gap: they are optimized on narrow candidate pools but must discriminate against hundreds of millions of items during inference. Furthermore, while transitioning to higher-capacity backbones can mitigate this gap, simply replacing a mature model can lead to inconsistent retrieval behavior and a loss of the domain-specific knowledge established in previous iterations. In this paper, we present a unified pipeline deployed at Walmart that addresses both signal quality and model evolution. Our contributions are two-fold: (1) Hybrid Hard Negative Mining: We integrate Online Cross-Batch Sampling to increase negative diversity by an order of magnitude and Hybrid Offline Mining, which combines cross-encoder predictions with metadata heuristics to identify nuanced mismatches. (2) Legacy-Aware Distillation: We transition from DistilBERT to a higher-capacity GTE-base encoder. To ensure a smooth and superior transition, we introduce a Warm-Start Distillation technique that transfers domain-specific expertise from the legacy model to the new backbone. Validated through extensive offline experiments and online A/B testing, the proposed pipeline is deployed in live production, delivering a +7.34% improvement in NDCG@5 and a +0.50% lift in gross revenue.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。