通过隐式用户画像模拟真实对话,提升对话系统评估与训练效果
Know You First and Be You Better: Modeling Human-Like User Simulators via Implicit Profiles
- 从人机交互中推断隐式用户特征,构建个性化对话模拟框架
- 在真实性、多样性上超越基线,对话一致性保持良好
- 适合用于大模型对话系统的自动评估与协同训练
用户模拟器对复现人类与对话系统交互至关重要,支持对话系统协同训练与自动评估,尤其适用于大语言模型(LLM)。现有角色扮演方法存在话语级真实感不足、用户级多样性差的问题,常受制于角色混淆和对知名人物预设档案的依赖。而直接文本模拟忽略了人格等隐含特质及对话层面的一致性。为此,我们提出隐式用户画像用户模拟器(USP),通过分析人机交互数据推断隐式用户特征,实现个性化且真实的对话模拟。首先构建基于LLM的提取器,采用全面的画像模板;随后通过条件监督微调与循环一致性强化学习,在话语与对话层级进行优化;最后引入多样化的画像采样器捕捉真实用户分布。实验表明,USP在真实性与多样性上优于强基线,一致性相当。此外,使用USP评估LLM在动态多轮任务中的表现,与主流基准高度一致,验证了其在实际应用中的有效性。
原文摘要 · Abstract (English)
User simulators are crucial for replicating human interactions with dialogue systems, supporting both collaborative training and automatic evaluation, especially for large language models (LLMs). However, current role-playing methods face challenges such as a lack of utterance-level authenticity and user-level diversity, often hindered by role confusion and dependence on predefined profiles of well-known figures. In contrast, direct simulation focuses solely on text, neglecting implicit user traits like personality and conversation-level consistency. To address these issues, we introduce the User Simulator with Implicit Profiles (USP), a framework that infers implicit user profiles from human-machine interactions to simulate personalized and realistic dialogues. We first develop an LLM-driven extractor with a comprehensive profile schema, then refine the simulation using conditional supervised fine-tuning and reinforcement learning with cycle consistency, optimizing at both the utterance and conversation levels. Finally, a diverse profile sampler captures the distribution of real-world user profiles. Experimental results show that USP outperforms strong baselines in terms of authenticity and diversity while maintaining comparable consistency. Additionally, using USP to evaluate LLM on dynamic multi-turn aligns well with mainstream benchmarks, demonstrating its effectiveness in real-world applications.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。