用问卷直接对齐用户满意度,让推荐模型更懂真实喜好。
Towards End-to-End Alignment of User Satisfaction via Questionnaire in Video Recommendation
- 分离问卷与行为数据路径,避免被海量行为信号淹没。
- 结合LoRA和强化学习,实现在线实时优化满意度。
- 已落地生产,显著提升用户满意度和业务指标。
短视频推荐系统通常基于点击、观看时长等密集行为信号优化排序模型,但这些信号仅为用户满意度的间接代理,常含噪声与偏差。近期通过问卷获取的显式满意度反馈虽质量高,但极为稀疏,易被海量行为数据覆盖,难以融入在线推荐模型。为此,我们提出端到端对齐问卷反馈的框架EASQ,实现推荐模型与真实用户满意度的实时对齐。具体地,通过多任务架构与轻量级LoRA模块构建独立参数路径,分离稀疏满意度监督与密集行为信号,防止前者被淹没;LoRA以参数隔离方式预注入偏好,确保主干稳定的同时优化满意度。此外,采用针对在线学习设计的DPO优化目标,实时对齐主模型输出与稀疏问卷信号。该设计支持端到端在线学习,使模型能持续适应新问卷反馈,同时保持主干稳定性与有效性。大量离线实验与大规模线上A/B测试表明,EASQ在多场景下持续提升用户满意度指标。该方案已在生产级短视频推荐系统中成功部署,带来显著且稳定的业务收益。
原文摘要 · Abstract (English)
Short-video recommender systems typically optimize ranking models using dense user behavioral signals, such as clicks and watch time. However, these signals are only indirect proxies of user satisfaction and often suffer from noise and bias. Recently, explicit satisfaction feedback collected through questionnaires has emerged as a high-quality direct alignment supervision, but is extremely sparse and easily overwhelmed by abundant behavioral data, making it difficult to incorporate into online recommendation models. To address these challenges, we propose a novel framework which is towards End-to-End Alignment of user Satisfaction via Questionaire, named EASQ, to enable real-time alignment of ranking models with true user satisfaction. Specifically, we first construct an independent parameter pathway for sparse questionnaire signals by combining a multi-task architecture and a lightweight LoRA module. The multi-task design separates sparse satisfaction supervision from dense behavioral signals, preventing the former from being overwhelmed. The LoRA module pre-inject these preferences in a parameter-isolated manner, ensuring stability in the backbone while optimizing user satisfaction. Furthermore, we employ a DPO-based optimization objective tailored for online learning, which aligns the main model outputs with sparse satisfaction signals in real time. This design enables end-to-end online learning, allowing the model to continuously adapt to new questionnaire feedback while maintaining the stability and effectiveness of the backbone. Extensive offline experiments and large-scale online A/B tests demonstrate that EASQ consistently improves user satisfaction metrics across multiple scenarios. EASQ has been successfully deployed in a production short-video recommendation system, delivering significant and stable business gains.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。