用分支对话结构提升大模型多轮问诊的诊断准确率
Conversation Forests: The Key to Fine Tuning Large Language Models for Multi-Turn Medical Conversations is Branching
- 设计分支对话架构，每轮生成多种可能延续
- 在模拟问诊中诊断准确率优于传统线性模型
- 适合需要理解对话动态的医疗问答场景
直接偏好优化（DPO）和组相对策略优化（GRPO）等微调方法在单轮任务中表现良好，但在多轮应用如诊断问诊中效果有限，因无法捕捉早期对话对后续结果的影响。为弥补这一不足，本文提出萨维奇对话森林（SCF），一种基于分支对话架构的强化学习框架。SCF在每轮对话中生成多个可能的后续发展路径，使模型学习不同初始回应如何影响下游交互与诊断结果。在模拟医生-患者对话的实验中，带分支的SCF在诊断准确性上显著优于线性对话架构。研究假设其优势源于跨轮次间更丰富、相互依赖的训练信号。结果表明，分支式训练架构是复杂多轮对话任务中微调大语言模型的关键策略。
原文摘要 · Abstract (English)
Fine-tuning methods such as Direct Preference Optimization (DPO) and Group Relative Policy Optimization (GRPO) have demonstrated success in training large language models (LLMs) for single-turn tasks. However, these methods fall short in multi-turn applications, such as diagnostic patient interviewing, where understanding how early conversational turns influence downstream completions and outcomes is essential. In medicine, a multi-turn perspective is critical for learning diagnostic schemas and better understanding conversation dynamics. To address this gap, I introduce Savage Conversation Forests (SCF), a reinforcement learning framework that leverages a branched conversation architecture to fine-tune LLMs for multi-turn dialogue. SCF generates multiple possible conversation continuations at each turn, enabling the model to learn how different early responses affect downstream interactions and diagnostic outcomes. In experiments simulating doctor-patient conversations, SCF with branching outperforms linear conversation architectures on diagnostic accuracy. I hypothesize that SCF's improvements stem from its ability to provide richer, interdependent training signals across conversation turns. These results suggest that a branched training architecture is an important strategy for fine tuning LLMs in complex multi-turn conversational tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。