电商搜索中兼顾语义相关性与用户参与度,避免推荐热门商品的偏见。
Joint Optimization of Relevance and Engagement in Multi-Task Ranking for E-Commerce with Efficient LLM Supervision

- 用序数相关性头建模语义匹配,支持可控权衡相关性与点击行为。
- 通过轻量级大模型生成三档相关性标签,实现超亿级样本高效标注。
- 线上实验验证显著提升语义准确性,同时保持核心转化指标稳定。
仅优化用户参与度信号的工业搜索排序模型易引入系统性偏差,倾向于推荐热门或低价商品,忽视用户真实语义意图。我们提出一个生产规模的多任务排序系统,将语义相关性作为核心优化目标,实现相关性与参与度的显式可控平衡。架构采用序数相关性头,预测在不同相关性阈值下的累积概率,保留标签固有顺序。该输出与参与度头通过统一价值模型评分函数融合,实现语义质量与短期行为信号的系统性权衡。为提供高质量监督信号,我们使用微调后的轻量级大语言模型生成三类序数相关性标签:无关、中等相关、高度相关。针对标签分布敏感性问题,确保与人工标注高度一致,实现超过1亿个查询-商品对的高效标注。离线评估(包括NDCG@10)与在线A/B实验均表明,该方法显著提升语义对齐度,同时维持核心参与度目标。
原文摘要 · Abstract (English)
Optimizing industrial search ranking models solely for user engagement signals often introduces systematic biases, prioritizing popular or price-anchored items that may not satisfy semantic intent. We present a production-scale multi-task ranking system that integrates semantic relevance as a primary optimization objective, enabling explicit and controllable relevance-engagement trade-offs. Our architecture employs an ordinal relevance head that predicts cumulative probabilities over relevance thresholds, preserving the inherent ordering of labels. These outputs are integrated with engagement heads through a unified value model scoring function, enabling systematic balancing of semantic quality and short-term behavioral signals. To provide high-quality supervision for this multi-task framework, we utilize fine-tuned lightweight Large Language Models (LLMs) to generate three-level ordinal relevance labels: irrelevant, moderately relevant, and highly relevant. We address challenges regarding label distribution sensitivity and ensure high alignment with human annotations to enable efficient labeling for over 100 million query-item pairs. Evaluation across offline metrics, including NDCG@10, and online A/B experiments demonstrates that our approach significantly improves semantic alignment while preserving core engagement objectives.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。