用大模型评估内容质量,解决长尾视频搜索中低质内容泛滥问题。
Unbiased Multimodal Reranking for Long-Tail Short-Video Search
- 利用大模型构建多模态标注数据,无需真实用户行为
- 两阶段训练提升排序准确性,显著改善长尾查询结果
- 线上测试显示用户体验和播放量双提升,适合长尾搜索场景
快手每日处理数亿次视频搜索,但长尾查询因用户行为数据稀疏,导致模型偏好低质内容如标题党与浅层内容。近期大语言模型(LLM)凭借其世界知识,可不依赖用户交互评估内容质量。为此,我们提出一种基于LLM的多模态重排框架:第一阶段利用多模态证据生成高质量标注用于监督微调;第二阶段引入成对偏好优化,学习候选内容间的部分排序关系。推理时,通过生成的经验评分促进高质量但曝光不足的视频被重排,并结合强化学习优化页面级推荐。离线实验表明,该方法在AUC、NDCG@K及人工偏好判断上均优于强基线;在线A/B测试覆盖15%流量,用户体验与消费指标同步提升,验证了其在长尾视频搜索中的实际价值。
原文摘要 · Abstract (English)
Kuaishou serving hundreds of millions of searches daily, the quality of short-video search is paramount. However, it suffers from a severe Matthew effect on long-tail queries: sparse user behavior data causes models to amplify low-quality content such as clickbait and shallow content. The recent advancements in Large Language Models (LLMs) offer a new paradigm, as their inherent world knowledge provides a powerful mechanism to assess content quality, agnostic to sparse user interactions. To this end, we propose a LLM-driven multimodal reranking framework, which estimates user experience without real user behavior. The approach involves a two-stage training process: the first stage uses multimodal evidence to construct high-quality annotations for supervised fine-tuning, while the second stage incorporates pairwise preference optimization to help the model learn partial orderings among candidates. At inference time, the resulting experience scores are used to promote high-quality but underexposed videos in reranking, and further guide page-level optimization through reinforcement learning. Experiments show that the proposed method achieves consistent improvements over strong baselines in offline metrics including AUC, NDCG@K, and human preference judgement. An online A/B test covering 15\% of traffic further demonstrates gains in both user experience and consumption metrics, confirming the practical value of the approach in long-tail video search scenarios.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。