用评分标准逐步训练大模型，让医疗对话更精准可靠。
InfiMed-ORBIT: Aligning LLMs on Open-Ended Complex Tasks via Rubric-Based Incremental Training
- 基于动态生成的评分标准，分步优化模型对话能力。
- 仅用2000样本，医疗问答得分从7.0提升至27.5。
- 适合医疗对话、需要高可信度的开放任务场景。
强化学习（RL）在代码生成等可自动评估的任务中表现优异，但在开放性医疗对话中因反馈模糊、依赖上下文且难以量化为单一奖励信号而效果受限，常需大量人工标注的奖励模型，易引发奖励黑客问题。为此，我们提出ORBIT框架，一种面向关键医疗对话的开放式评分标准渐进式训练方法。该框架将医疗对话构建与动态生成的病例相关评分标准结合，作为增量强化学习的自适应指导。不同于依赖外部医学知识库或手工规则的方法，ORBIT通过评分标准引导评估，可直接使用通用指令遵循型大模型实现，无需特定任务的判别器微调。仅使用2000个训练样本，即把Qwen3-4B-Instruct在HealthBench-Hard上的得分从7.0提升至27.5，在同类开源模型中达到领先水平，且随着评分标准覆盖范围扩大，咨询质量持续增强。
原文摘要 · Abstract (English)
Reinforcement learning (RL) has powered many recent breakthroughs in large language models (LLMs), especially for tasks where rewards can be computed automatically, such as code generation. However, it is less effective in open-ended medical dialogue, where feedback is ambiguous, context-dependent, and difficult to simply summarize into a single scalar signal-often requiring heavily supervised reward models and creating risks of reward hacking. Thus, we introduce ORBIT, an open-ended rubric-based incremental training framework tailored for critical medical dialogues. ORBIT integrates medical dialogue construction with dynamically generated case-conditioned rubrics that serve as adaptive guides for incremental RL. Unlike approaches that rely on external medical knowledge bases or handcrafted rules, ORBIT uses rubric-guided evaluation and can be implemented with general-purpose instruction-following LLMs, avoiding task-specific judge fine-tuning. With only 2k training samples, ORBIT raises Qwen3-4B-Instruct's HealthBench-Hard score from 7.0 to 27.5, achieving state-of-the-art performance among similarly sized open-source models while maintaining strong consultation quality as rubric coverage broadens.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。