用随机采样优化推荐系统中的偏好学习，提升排序效果。
Aligning Multimodal Sequential Recommendations via Robust Direct Preference Optimization with Sparse MoE
- 用动态候选池随机采样替代固定负样本，缓解误判干扰。
- 在三个亚马逊数据集上实现最高5.25%的NDCG@5提升。
- 结合稀疏专家模型，性能提升但推理成本几乎不变。
基于偏好的对齐目标已在大语言模型的强化学习人类反馈（RLHF）和推荐系统中广泛应用。然而，现有研究较少探讨直接偏好优化（DPO）在隐式反馈场景下的表现，其中未观察到的物品并非可靠负样本。我们在多模态序列推荐任务上系统比较了常见的负样本选择策略及其与DPO训练的交互效果。核心发现是：将确定性的硬负样本替换为从动态顶K候选池中进行随机采样，能持续提升排序性能。其有效性归因于两点：(1) 减少由错误负样本引起的抑制性梯度；(2) 在保留有效硬信号的同时，通过可控随机性平滑优化过程。结合可选的稀疏专家（MoE）编码器以实现高效容量扩展，RoDPO在三个Amazon基准上实现高达5.25%的NDCG@5提升，且推理成本几乎不变。
原文摘要 · Abstract (English)
Preference-based alignment objectives have been widely adopted, from RLHF-style pairwise learning in large language models to emerging applications in recommender systems. Yet, existing work rarely examines how Direct Preference Optimization (DPO) behaves under implicit feedback, where unobserved items are not reliable negatives. We conduct systematic experiments on multimodal sequential recommendation to compare common negative-selection strategies and their interaction with DPO training. Our central finding is that a simple modification, replacing deterministic hard negatives with stochastic sampling from a dynamic top-K candidate pool, consistently improves ranking performance. We attribute its effectiveness to two factors: (1) reducing erroneous suppressive gradients caused by false negatives, and (2) retaining informative hard signals while smoothing optimization via controlled stochasticity. With an optional sparse Mixture-of-Experts encoder for efficient capacity scaling, RoDPO achieves up to 5.25% NDCG@5 on three Amazon benchmarks, with nearly unchanged inference cost.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。