对比11种解码策略,发现确定性方法在医疗文本生成中更优。
A Comparative Study of Decoding Strategies in Medical Text Generation
- 采用11种解码策略,对比其在5类医疗任务中的表现。
- 束搜索得分最高,随机采样表现最差,大模型推理更慢但未必更稳。
- 医疗专用模型并非全面领先,解码方式影响甚至超过模型选择。
大型语言模型依赖不同解码策略生成文本,而这些选择可能显著影响输出质量。在医疗领域,准确至关重要,但解码策略的影响仍研究不足。本文在五项开放式医疗任务(包括翻译、摘要、问答、对话和图像描述)中,评估了11种解码策略在医学专用与通用大模型上的表现,模型规模各异。结果表明,确定性策略普遍优于随机策略:束搜索得分最高,η采样和top-k采样表现最差。较慢的解码方法通常产生更高质量输出。大模型整体得分更高,但推理时间更长,且对解码策略不具更强鲁棒性。令人意外的是,尽管医学专用模型在两项任务中表现更优,统计分析显示其无整体优势,并表现出对解码选择更大的敏感性。进一步比较多种评估指标发现,相关性因任务而异,MAUVE与BERTScore、ROUGE的相关性弱,且对解码策略更敏感。这些结果凸显在医疗应用中需谨慎选择解码方法,其影响有时甚至超过模型选择本身。
原文摘要 · Abstract (English)
Large Language Models (LLMs) rely on various decoding strategies to generate text, and these choices can significantly affect output quality. In healthcare, where accuracy is critical, the impact of decoding strategies remains underexplored. We investigate this effect in five open-ended medical tasks, including translation, summarization, question answering, dialogue, and image captioning, evaluating 11 decoding strategies with medically specialized and general-purpose LLMs of different sizes. Our results show that deterministic strategies generally outperform stochastic ones: beam search achieves the highest scores, while η and top-k sampling perform worst. Slower decoding methods tend to yield better quality. Larger models achieve higher scores overall but have longer inference times and are no more robust to decoding. Surprisingly, while medical LLMs outperform general ones in two of the five tasks, statistical analysis shows no overall performance advantage and reveals greater sensitivity to decoding choice. We further compare multiple evaluation metrics and find that correlations vary by task, with MAUVE showing weak agreement with BERTScore and ROUGE, as well as greater sensitivity to the decoding strategy. These results highlight the need for careful selection of decoding methods in medical applications, as their influence can sometimes exceed that of model choice.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。