用潜在变量控制对话长期行为，让聊天机器人更懂情绪、更会策略性对话。
SAGE: Steering Dialog Generation with Future-Aware State-Action Augmentation
- 引入状态-动作链，用潜在变量捕捉情绪与策略
- 生成式控制使对话自然且可调，性能优于基准模型
- 适合想构建智能对话系统的研究者与开发者
大型语言模型在任务导向应用中表现优异，但构建能进行自然、战略性对话的情绪智能聊天机器人仍具挑战。我们提出SAGE方法，通过潜在变量控制对话生成的长期行为。核心是状态-动作链（SAC），在标准微调基础上引入蕴含对话回合间情绪状态与策略的潜在变量。推理时，这些变量在每轮回复前生成，实现对对话进程的粗粒度控制，同时保持自然交互模式。我们还设计了自提升流程，结合对话树搜索、大模型奖励建模和针对性微调，优化对话轨迹。实验表明，该方法在情感智能指标上表现更好，同时在主流LLM基准测试中保持强性能。潜在变量的离散特性支持基于搜索的策略，并为未来对话系统的强化学习奠定基础，学习可发生在状态层面而非词元层面。
原文摘要 · Abstract (English)
Recent advances in large language models have demonstrated impressive capabilities in task-oriented applications, yet building emotionally intelligent chatbots that can engage in natural, strategic conversations remains a challenge. We present a novel approach called SAGE that uses latent variables to control long-horizon behavior in dialogue generation. At the core of our method is the State-Action Chain (SAC), which augments standard language model fine-tuning by introducing latent variables that encapsulate emotional states and conversational strategies between dialogue turns. During inference, these variables are generated before each response, enabling coarse-grained control over dialogue progression while maintaining natural interaction patterns. We also introduce a self-improvement pipeline that leverages dialogue tree search, LLM-based reward modeling, and targeted fine-tuning to optimize conversational trajectories. Our experimental results show that models trained with this approach demonstrate improved performance in emotional intelligence metrics while maintaining strong capabilities on LLM benchmarks. The discrete nature of our latent variables facilitates search-based strategies and provides a foundation for future applications of reinforcement learning to dialogue systems, where learning can occur at the state level rather than the token level. https://github.com/apple/ml-sage-dialog-gen
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。