用用户愿不愿回来评估内容价值,比点击更准预测留存。
Retentive Relevance: Capturing Long-Term User Value in Recommendation Systems
- 设计问卷直接测量用户未来回访意愿,捕捉长期价值
- 在真实平台测试中显著优于点击率和短期满意度
- 适合做推荐系统优化,尤其对新用户或低活跃用户
推荐系统长期依赖点击、点赞等短期行为信号,但这些信号噪声大、稀疏且难以反映长期满意度与留存。本文提出「Retentive Relevance」——一种基于调查的、面向内容级别的用户回访意图度量方法,聚焦未来行为意向而非即时感受,更准确预测用户留存。通过心理测量学方法验证其收敛效度、区分效度及行为效度。大规模离线建模显示,该指标在预测次日留存上显著优于传统互动信号和其他调查指标,尤其对历史行为少的用户表现更优。我们构建了可投入生产的代理模型,将其集成到社交媒体平台多阶段排序系统的最终阶段。校准后的评分提升显著改善了用户参与度与留存,同时减少低质内容曝光。大规模A/B实验验证了效果。本研究首次在生产系统中建立内容感知与用户留存之间的实证框架,提供了一种可扩展、以用户为中心的解决方案,推动平台增长与体验优化,并为负责任的人工智能发展提供启示。
原文摘要 · Abstract (English)
Recommendation systems have traditionally relied on short-term engagement signals, such as clicks and likes, to personalize content. However, these signals are often noisy, sparse, and insufficient for capturing long-term user satisfaction and retention. We introduce Retentive Relevance, a novel content-level survey-based feedback measure that directly assesses users' intent to return to the platform for similar content. Unlike other survey measures that focus on immediate satisfaction, Retentive Relevance targets forward-looking behavioral intentions, capturing longer term user intentions and providing a stronger predictor of retention. We validate Retentive Relevance using psychometric methods, establishing its convergent, discriminant, and behavioral validity. Through large-scale offline modeling, we show that Retentive Relevance significantly outperforms both engagement signals and other survey measures in predicting next-day retention, especially for users with limited historical engagement. We develop a production-ready proxy model that integrates Retentive Relevance into the final stage of a multi-stage ranking system on a social media platform. Calibrated score adjustments based on this model yield substantial improvements in engagement, and retention, while reducing exposure to low-quality content, as demonstrated by large-scale A/B experiments. This work provides the first empirically validated framework linking content-level user perceptions to retention outcomes in production systems. We offer a scalable, user-centered solution that advances both platform growth and user experience. Our work has broad implications for responsible AI development.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。