提升搜索推荐效果,解决冷启动与曝光偏差问题。
PRECTR-V2:Unified Relevance-CTR Framework with Cross-User Preference Mining, Exposure Bias Correction, and LLM-Distilled Encoder Optimization
- 通过全局偏好挖掘,增强低活跃用户的个性化匹配能力。
- 用噪声注入生成难负样本,纠正高相关性曝光带来的偏差。
- 用大模型蒸馏训练轻量编码器,更好适配点击率预测。
在搜索系统中,有效协调搜索相关性匹配与点击率(CTR)预测两大核心目标,对于发现用户兴趣并提升平台收益至关重要。此前的PRECTR框架虽实现了两任务统一,但仍面临三大挑战:一是低活跃用户和新用户行为数据稀疏,难以实现有效的个性化相关性建模;二是排序模型训练数据主要来自高相关性曝光,与粗排候选集分布不一致,导致泛化偏差;三是受延迟约束,原模型采用嵌入+MLP架构并冻结BERT编码器,无法联合优化,使表征学习与CTR微调存在错位。为此,我们提出PRECTR-V2:首先,通过查询特定的全局相关性偏好挖掘,缓解低活跃用户的行为稀疏问题;其次,利用嵌入噪声注入与相关性标签重构构建难负样本,并通过成对损失优化其相对于正样本的排序,以修正曝光偏差;最后,基于大语言模型(LLM)和监督微调(SFT)在文本相关性分类任务上进行知识蒸馏,预训练轻量级Transformer编码器,替代原有冻结的BERT模块,支持联合优化,突破传统Emb+MLP范式。
原文摘要 · Abstract (English)
In search systems, effectively coordinating the two core objectives of search relevance matching and click-through rate (CTR) prediction is crucial for discovering users' interests and enhancing platform revenue. In our prior work PRECTR, we proposed a unified framework to integrate these two subtasks,thereby eliminating their inconsistency and leading to mutual benefit.However, our previous work still faces three main challenges. First, low-active users and new users have limited search behavioral data, making it difficult to achieve effective personalized relevance preference modeling. Second, training data for ranking models predominantly come from high-relevance exposures, creating a distribution mismatch with the broader candidate space in coarse-ranking, leading to generalization bias. Third, due to the latency constraint, the original model employs an Emb+MLP architecture with a frozen BERT encoder, which prevents joint optimization and creates misalignment between representation learning and CTR fine-tuning. To solve these issues, we further reinforce our method and propose PRECTR-V2. Specifically, we mitigate the low-activity users' sparse behavior problem by mining global relevance preferences under the specific query, which facilitates effective personalized relevance modeling for cold-start scenarios. Subsequently, we construct hard negative samples through embedding noise injection and relevance label reconstruction, and optimize their relative ranking against positive samples via pairwise loss, thereby correcting exposure bias. Finally, we pretrain a lightweight transformer-based encoder via knowledge distillation from LLM and SFT on the text relevance classification task. This encoder replaces the frozen BERT module, enabling better adaptation to CTR fine-tuning and advancing beyond the traditional Emb+MLP paradigm.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。