提出分层序数排序框架,提升电商搜索相关性判断准确率
CORE: A Unified Cascaded Ordinal Relevance Estimation Framework for E-commerce Search
- 将相关性评估转为分层二分类决策,按等级顺序逐步判断
- 在线实验显示劣质案例率降低15.94%,效果显著优于传统方法
- 适配大模型与线上BERT模型,兼顾精度与实时性
电商搜索中的相关性排序是影响排名质量与用户体验的核心任务。尽管本质上属于序数分类问题,但通常被当作普通多分类处理,忽视了相关性等级间的自然顺序,对相邻与远距离误判施加相同惩罚,导致学习目标不匹配。为此,我们提出一种统一的级联二分类框架,适用于大语言模型推理与线上BERT模型,将相关性估计重构为逐级决策过程,把多分类预测分解为从高到低的相关性层级的一系列有序二分类判断。针对大语言模型,设计分步推理流程,结合剪枝策略与层级专属奖励函数;针对线上BERT模型,以多层级二分类头替代传统分类头,并将大模型能力蒸馏至线上模型。大规模离线工业基准测试与线上A/B实验表明,该框架显著提升相关性表现,线上劣质案例率下降15.94%。进一步分析表明,层级建模对相关性评估具有显著有效性。
原文摘要 · Abstract (English)
Ranking relevance is a fundamental task in e-commerce search, directly affecting ranking quality and consumer experience. Although inherently an ordinal classification problem, it is commonly formulated as conventional multi-class classification, which overlooks the natural order among relevance levels and assigns equal penalties to adjacent and distant misclassifications. This mismatch leads to suboptimal learning objectives for practical relevance evaluation. To address this issue, we propose a unified cascaded binary classification framework applicable to both large language model inference and online BERT-based inference, which reformulates relevance estimation as a sequential decision process and decomposes multi-class prediction into a series of ordered binary judgments from higher to lower relevance tiers. For large language models, we design a step-wise reasoning procedure with pruning strategies and tier-specific reward functions. For the online BERT model, we replace the conventional classification head with multiple level-wise binary classifiers and distill the capabilities of large language models into the online model. Extensive offline industrial benchmark evaluations and online A/B experiments demonstrate that the proposed framework substantially improves relevance performance, reducing the online bad-case rate by 15.94\%. Further analyses suggest that tier-wise modeling is effective for relevance estimation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。