让聊天机器人模拟用户时，能持续追踪目标并保持一致行为。
Goal Alignment in LLM-Based User Simulators for Conversational AI
- 用状态追踪技术实时记录用户目标变化
- 在两个基准上显著提升目标对齐度
- 适合做对话系统训练与评估的研究者
用户模拟器对对话式AI至关重要，可通过模拟交互实现智能体的规模化开发与评估。尽管当前大语言模型（LLMs）已具备较强用户模拟能力，但我们在多轮对话中发现其难以持续表现出目标导向行为，这一关键缺陷影响了下游应用的可靠性。为此，我们提出用户目标状态追踪（UGST）框架，用于在对话过程中跟踪用户目标进展。基于UGST，我们构建了三阶段方法论，使模拟器能自主追踪目标演变并生成与目标一致的回复。此外，我们建立了全面的评估指标体系，用于衡量用户模拟器的目标对齐程度，并在两个基准数据集（MultiWOZ 2.4 和 τ-Bench）上验证了该方法的显著提升效果。本工作填补了对话式AI中的关键空白，确立了UGST作为构建目标对齐用户模拟器的核心框架。
原文摘要 · Abstract (English)
User simulators are essential to conversational AI, enabling scalable agent development and evaluation through simulated interactions. While current Large Language Models (LLMs) have advanced user simulation capabilities, we reveal that they struggle to consistently demonstrate goal-oriented behavior across multi-turn conversations--a critical limitation that compromises their reliability in downstream applications. We introduce User Goal State Tracking (UGST), a novel framework that tracks user goal progression throughout conversations. Leveraging UGST, we present a three-stage methodology for developing user simulators that can autonomously track goal progression and reason to generate goal-aligned responses. Moreover, we establish comprehensive evaluation metrics for measuring goal alignment in user simulators, and demonstrate that our approach yields substantial improvements across two benchmarks (MultiWOZ 2.4 and τ-Bench). Our contributions address a critical gap in conversational AI and establish UGST as an essential framework for developing goal-aligned user simulators.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。