通过多智能体自动挖掘有效数据并校验标签，让搜索相关性模型持续自我进化。
SERM: Self-Evolving Relevance Model with Agent-Driven Learning from Massive Query Streams
- 用多智能体系统识别查询分布变化，主动筛选高价值训练样本。
- 通过双层一致机制生成可靠伪标签，提升模型迭代质量。
- 在日均数十亿请求的工业场景中验证，显著提升搜索相关性表现。
由于真实世界查询流具有动态演变特性，相关性模型难以泛化至实际搜索场景。虽然自演化技术是潜在解决方案，但在大规模工业场景下仍面临两大挑战：(1) 有效样本稀疏且难识别；(2) 当前模型生成的伪标签可能不可靠。为此，本文提出自演化相关性模型（SERM），包含两个互补的多智能体模块：多智能体样本挖掘器，用于检测分布偏移并识别有信息量的训练样本；多智能体相关性标注器，通过两级一致性框架提供可靠标签。我们在日均服务数十亿用户请求的大规模工业环境中评估 SERM，实验结果表明，通过迭代自演化，模型在离线多语言评估和在线测试中均实现显著性能提升。
原文摘要 · Abstract (English)
Due to the dynamically evolving nature of real-world query streams, relevance models struggle to generalize to practical search scenarios. A sophisticated solution is self-evolution techniques. However, in large-scale industrial settings with massive query streams, this technique faces two challenges: (1) informative samples are often sparse and difficult to identify, and (2) pseudo-labels generated by the current model could be unreliable. To address these challenges, in this work, we propose a Self-Evolving Relevance Model approach (SERM), which comprises two complementary multi-agent modules: a multi-agent sample miner, designed to detect distributional shifts and identify informative training samples, and a multi-agent relevance annotator, which provides reliable labels through a two-level agreement framework. We evaluate SERM in a large-scale industrial setting, which serves billions of user requests daily. Experimental results demonstrate that SERM can achieve significant performance gains through iterative self-evolution, as validated by extensive offline multilingual evaluations and online testing.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。