让大模型学会动态信任外部信息,提升电商搜索精准度。
Learning to Trust: Dynamic Utilization of Retrieval-Augmented Generation for E-commerce Search Relevance
- 用强化学习教模型判断何时用外部知识,自动调节依赖程度。
- 在淘宝日均数十亿请求下,延迟低于400ms,关键指标持续提升。
- 无需人工标注,适配高并发、低延迟的工业级搜索系统。
准确评估查询与商品的相关性对电商排序和转化至关重要。尽管大语言模型(LLMs)擅长推理,但在长尾或快速变化的查询中常缺乏专业知识,需借助检索增强生成(RAG)。然而生产环境面临三大挑战:(1)外部上下文固有噪声且不一致;(2)极端延迟预算禁止多阶段处理;(3)模型必须在单一推理过程中同时评估相关性和上下文可信度。我们提出DyKnow-RAG,一种基于组相对策略优化(GRPO)的强化学习框架,通过参数化仅与含上下文双组滚动策略及后验驱动的组间优势缩放机制,使模型在无人工标注和额外推理开销下优化上下文使用。系统还集成结构化思维链(CoT)与不确定性优先的强化学习池以稳定训练。离线评估显示,在噪声敏感查询子集上宏观F1和准确率显著提升。更重要的是,DyKnow-RAG已在淘宝生产系统部署,服务数亿活跃用户及每日数十亿搜索请求。控制A/B测试表明关键业务指标(如GSB、Item Goodrate)持续增长,同时保持p99延迟低于400ms。该工作为大规模工业搜索中在严苛效率约束下操作噪声RAG提供了可扩展、可部署的范式。
原文摘要 · Abstract (English)
Accurately estimating query-item relevance is vital for e-commerce ranking and conversion. While Large Language Models (LLMs) excel at reasoning, they often lack specialized knowledge required for long-tail or fast-evolving queries, necessitating Retrieval-Augmented Generation (RAG). However, production environments face three critical challenges: (1) external context is inherently noisy and inconsistent; (2) extreme latency budgets prohibit multi-stage processing or refinement; and (3) the model must simultaneously assess relevance and context-trust within a unified inference pass. We propose DyKnow-RAG, a reinforcement learning framework that teaches LLMs to learn to trust through dynamic utilization of external knowledge. Built on Group Relative Policy Optimization (GRPO), DyKnow-RAG utilizes a dual-group rollout strategy (parametric-only vs. with-context) and a posterior-driven inter-group advantage scaling mechanism. This enables the model to optimize context utilization without human process labels or extra inference overhead. Our pipeline further integrates structured Chain-of-Thought (CoT) and an uncertainty-prioritized RL pool to stabilize training.Offline evaluations show significant Macro-F1 and Accuracy gains, particularly on noise-sensitive query slices. Importantly, DyKnow-RAG has been deployed in Taobao's production system, serving hundreds of millions of active users and billions of daily search requests. Controlled A/B tests demonstrate consistent lifts in key business metrics, including GSB and Item Goodrate, while maintaining a p99 latency under 400ms. This work provides a scalable and deployable paradigm for operationalizing noisy RAG under extreme efficiency constraints of large-scale industrial search.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。