提出三种新算法,让对话推荐系统更聪明地选关键词、判断何时该问。
Leveraging the Power of Conversations: Optimal Key Term Selection in Conversational Contextual Bandits
- 用平滑上下文增强探索,提升偏好学习效果
- 对话累计后悔值降低14.6%以上,理论最优性被证明
- 适合需要高效交互的个性化推荐场景
对话式推荐系统通过主动询问关键术语并利用用户反馈来学习个性化偏好。对话上下文老虎机(Conversational contextual bandits)是该领域的主流方法,旨在平衡利用与探索以优化偏好学习。然而现有方法存在两大缺陷:其一,关键词选择策略探索不足,难以充分挖掘用户偏好,导致估计不准确;其二,对话启动依赖固定规则,当偏好明确时仍频繁交互,偏好不确定时却错失机会。为此,本文提出三种新算法:CLiSK通过平滑关键词上下文增强探索,CLiME基于偏好不确定性自适应启动对话,CLiSK-ME融合二者。理论上,三者均达到关于时间范围T的上界O(√(dT log T)),优于已有方法;同时给出匹配下界Ω(√(dT)),表明算法近乎最小最大最优。在合成与真实数据集上的大量实验表明,所提方法在累积后悔值上至少提升14.6%。
原文摘要 · Abstract (English)
Conversational recommender systems proactively query users with relevant "key terms" and leverage the feedback to elicit users' preferences for personalized recommendations. Conversational contextual bandits, a prevalent approach in this domain, aim to optimize preference learning by balancing exploitation and exploration. However, several limitations hinder their effectiveness in real-world scenarios. First, existing algorithms employ key term selection strategies with insufficient exploration, often failing to thoroughly probe users' preferences and resulting in suboptimal preference estimation. Second, current algorithms typically rely on deterministic rules to initiate conversations, causing unnecessary interactions when preferences are well-understood and missed opportunities when preferences are uncertain. To address these limitations, we propose three novel algorithms: CLiSK, CLiME, and CLiSK-ME. CLiSK introduces smoothed key term contexts to enhance exploration in preference learning, CLiME adaptively initiates conversations based on preference uncertainty, and CLiSK-ME integrates both techniques. We theoretically prove that all three algorithms achieve a tighter regret upper bound of $O(\sqrt{dT\log{T}})$ with respect to the time horizon $T$, improving upon existing methods. Additionally, we provide a matching lower bound $Ω(\sqrt{dT})$ for conversational bandits, demonstrating that our algorithms are nearly minimax optimal. Extensive evaluations on both synthetic and real-world datasets show that our approaches achieve at least a 14.6% improvement in cumulative regret.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。