用期望确认理论优化多轮对话推荐，提升用户满意度。
Expectation Confirmation Preference Optimization for Multi-Turn Conversational Recommendation Agent
- 基于期望确认理论建模用户满意度演化过程。
- 相比现有方法，效率更高且显著提升推荐效果。
- 适合研究对话系统优化与个性化推荐的开发者。
大型语言模型（LLMs）的进展推动了对话式推荐代理（CRAs）的发展，但这些代理常生成短视响应，难以持续引导用户并满足其期望。尽管偏好优化能有效对齐模型与用户预期，但在多轮对话中成本高且表现不佳。为此，我们提出一种新型多轮偏好优化范式ECPO，利用期望确认理论显式建模多轮对话中用户满意度的演变，揭示不满的根本原因。这些原因可用于针对性优化不满意的回应，实现逐轮偏好优化。ECPO巧妙地消除了现有方法中巨大的采样开销，同时确保优化过程带来实质性改进。为支持ECPO，我们引入一个基于LLM的用户模拟器AILO，用于模拟用户反馈并在推荐过程中进行期望确认。实验结果表明，ECPO显著提升了CRAs的交互能力，在效率和有效性上均优于现有MTPO方法。
原文摘要 · Abstract (English)
Recent advancements in Large Language Models (LLMs) have significantly propelled the development of Conversational Recommendation Agents (CRAs). However, these agents often generate short-sighted responses that fail to sustain user guidance and meet expectations. Although preference optimization has proven effective in aligning LLMs with user expectations, it remains costly and performs poorly in multi-turn dialogue. To address this challenge, we introduce a novel multi-turn preference optimization (MTPO) paradigm ECPO, which leverages Expectation Confirmation Theory to explicitly model the evolution of user satisfaction throughout multi-turn dialogues, uncovering the underlying causes of dissatisfaction. These causes can be utilized to support targeted optimization of unsatisfactory responses, thereby achieving turn-level preference optimization. ECPO ingeniously eliminates the significant sampling overhead of existing MTPO methods while ensuring the optimization process drives meaningful improvements. To support ECPO, we introduce an LLM-based user simulator, AILO, to simulate user feedback and perform expectation confirmation during conversational recommendations. Experimental results show that ECPO significantly enhances CRA's interaction capabilities, delivering notable improvements in both efficiency and effectiveness over existing MTPO methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。