用轻量门控机制实现广告检索中精准回溯与新类发现的平衡。
SMART: LLM-Augmented Hybrid Retrieval for Dynamic Product Ads
- 分路处理：规则查询用于精准匹配，LLM生成查询用于探索新类目。
- 仅10%用户触发LLM路径，实现90%推理成本降低。
- 在Snap实测中转化率提升27.6%，兼顾召回与效率。
动态商品广告需从数百万商品中检索相关商品，需平衡重定向（重新展示已知兴趣）与探查（发现新类目）两大目标。尽管大语言模型（LLMs）比传统嵌入模型更擅长捕捉语义意图，但大规模部署带来高昂推理成本和词汇不匹配问题。通过对数百万用户的控制实验，我们发现关键的检索分解：规则生成查询在词法BM25索引上擅长重定向，而LLM生成查询在密集向量ANN索引上更优。基于此，我们提出SMART（SeMantic-aware Adaptive ReTrieval）。通过轻量级质量门控识别初始关键词结果中的覆盖缺口，仅将约10%的用户自适应路由至LLM路径。离线评估显示，该门控策略在保持重定向性能的同时，以90%的降低实现接近完整的语义探查收益。最终，在Snap为期两周的在线A/B测试中，SMART相较强基线模型广告转化率提升+27.6%。
原文摘要 · Abstract (English)
Dynamic Product Ads (DPA) require retrieving relevant items from multi-million product catalogs, balancing two competing objectives: retargeting (re-surfacing known interests) and prospecting (discovering new categories). While Large Language Models (LLMs) capture semantic intent better than traditional embedding models, deploying them at scale introduces prohibitive inference costs and lexical mismatch issues. Through controlled experiments on millions of users, we demonstrate a critical retrieval decomposition: rule-generated queries excel at retargeting on a lexical BM25 index, while LLM-generated queries excel at prospecting on a dense ANN index. Building on this, we propose SMART (SeMantic-aware Adaptive ReTrieval). To manage costs, a lightweight quality gate identifies coverage gaps in initial keyword results, adaptively routing only the ~10% of users who benefit from semantic prospecting to the LLM path. Offline evaluation demonstrates that this gated approach captures the bulk of semantic prospecting gains in Relevance Score while maintaining competitive re-targeting performance at a 90% reduction in LLM costs. Finally, in a 2-week online A/B test at Snap, SMART improved the ad conversion rate by +27.6% over a strong embedding-based baseline.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。